Mar 172015
 

What if I were to tell you that the e-commerce site you’ve been developing could be hosted on a static website that would offer it premier performance? Or that the document management system could be run on any plain, old static server like Apache, nginx? Stick with me and we’ll explore this topic. This information may be too basic for advanced developers, but for most managers, this introduction may be long overdue.

A bit of history

Long ago, most websites were static. All they served were pages – html, text, and images. Hyperlinks allowed us to navigate from one page to another. Most were content-rich websites containing information, entertainment, and photos. Now it seems unthinkable that clicking on the hyperlinks was the only way to interact with the sites.

image

As shown in the picture, the protocol between the web server and the user is fixed: http protocol. The way the webserver interacts with backend information (in the case of static web, it is html, css, js, images and such) is up to us. A static website serves this information straight out of the file system.

As the readers began wanting to navigate sites in interesting ways or interact by adding content with the websites (like comments), we began to need server side programming. Serving out of a file system won’t do. The application needs to understand complex requests and somehow use the information in the files (or databases) to serve the information. Since database is a popular choice to keep structured information, web started serving as interfaces to the database – types of simple crud operations. In fact, applications like phpMyAdmin even let people manage the databases on the web.

Coming back to the web applications I mentioned, — e-commerce, digital marketing, document management systems — all these websites are considered interactive, dynamic, and programmable, and therefore need a web application server. The moment you choose one platform (typically it means an app server, a language, a framework, a set of libraries), you’re stuck with it forever.

Let’s see how we can design our way out.

Static web with automated look and feel support

Let us consider a simple case of information presentation. In this setup, all the consumers want to do is read the information. In that case, why not stick to a static website?

We can. The advantages are obvious. First and foremost, a static website is simple. It does not require any moving parts. Any old webserver can serve static content. You can even run a static website, with a simple one-liner in Python (python –mSimpleHTTPServer in Python 2.7 or python3 –m http.server).

It is also incredibly fast. The server has to simply send pages – it does not have to do any other computation. As such, it can be sped up even more by caching the content locally, or even using CDN (content delivery networks). You can use special purpose webservers such as nginx, that are optimized for serving static content too.

Furthermore, it is simple to maintain. You can back it up in the cloud, move to a different provider, and even serve it from your own desktop. You do not need to back up a database or worry about software upgrades, portability issues, or even support.

image

But, what are the downsides? Let us consider the problems:

  1. How do you create nice looking pages? Let us say that you want to create a standard look and feel. If you are the kind that use hand-coded HTML, you will cut and paste the HTML template and then edit it. Pretty soon, it becomes complex. If you want to change the look and feel, you will have to go through all the pages and edit the few lines.
  2. How do you create standard elements of a website? For example, you want a menu scheme. If you add a new area, then you have to go to each page and add a link in the menu to this new area of website. Or, if you want to create a sitemap, you will find yourself constantly updating the sitemap.

Considering that a website has repetitive information depending on the context, static way of maintaining a website is cumbersome. That is why, even for the static websites, people prefer WordPress, Joomla, or Drupal – all of which are meant for dynamically creating websites.

image

In these systems of dynamic content generation, the content is generated based on the request. No matter what application you use, it will have following issues:

  1. The application becomes slow: The server needs to execute the program to generate a page and serve the page. Compare it to the earlier scenario where the server merely had to read the file from the file system and serve! In fact, the server can even cache it, if the file doesn’t change often.
  2. The application becomes complex: All the flexibility of combining content with the themes based on configuration comes at a price. The application server has to do many things.

Consider the following full description of the stack. imageAny web server that is generating pages dynamically, needs to depend on an execution engine. It can be an engine that interprets a language, or executes a binary. For any language, we need a framework. For instance, if we are using Javascript, we can use Express as the framework. This framework can do many things. At a bare minimum, it can route the calls – that is, it interprets any request and maps the request to the right response. Next, it needs to have libraries that deal with writing the logic to actually manipulate the data. To present the data, we need a templating engine.

Of course, you have fancy names like Model (the data), view (the templates etc), and the controller (the routing) for this kind of framework (MVC).

The problem with any stack is that once you are committed to it, it is incredibly difficult to change it. To make matters worse, the frameworks that let you start easily (for example, lot of php based frameworks) are not the best frameworks for enterprise strength requirements (like say Java, which is excellent in libraries that deal with lot of integration). Ask Facebook! They started with Php and had to lot of optimizations to get the performance they want.

How can we do better? Can we still use the static website, and support better usability of the website? Reduce the costs of maintenance? The answer is yes. If we were to separate the content generation and content delivery, we can get best of the both worlds.

image

From this activity, what did we gain?

  1. We gained performance: The runtime has to serve static files, giving us several different options to improve the performance.
  2. We gained simplicity, somewhat: Technically, it is possible to mix and match different systems of generation (which is somewhat like using different languages and compiling down to machine code).

Readers can observe that content delivery networks can do this job of caching generated content on-the-fly, giving us the illusion of static generation of content.

Static website generators

If we are going to generate static website, what options do we have? Let us see the alternatives.

  1. Firstly, we can use the same dynamic content generators and dump out the entire site in static pages.  That is, if we are using Drupal, we can dump all the nodes. Or, with WordPress, or any other content management site.

    The challenge is, what if there is no unique URL for a content? Lot of content management sites offer multiple ways of accessing the static content. For example, Lotus notes was notorious for such URL generation. Then, the static content dumping can be difficult. More over, these systems are not meant for static website generation – the limitations keep showing up as you start relying on them.

  2. Secondly, we can use WYSIWYG editors such as Dreamweaver. They can create static content, apply a theme, and generate the website. They come with several beautiful themes, icons, and support scripts as well.

    The challenge is that these systems are not programmable. Suppose you are generating content from external system. These systems do not provide a way to automate the content ingestion and upgrading of the website.

  3. Thirdly, we can use a programmable system that generates the content. These days, this is the favored approach. These systems generate or update the complete website just from text based files. You can programmatically import content, update the website and publish it to production – all through scripting. Furthermore, they offer standard templating, support for CSS and Javascript libraries.

    The downside, of course, is that these systems are not mature. They are not general purpose either. Still, for an experienced programmer, this is a wonderful option.

There are several examples of  the third type of generation systems. The most popular ones are the ones that support blogging. For instance, Jekyll is a popular static website generator, written in Ruby, with a blog flavor. The content is authored in markdown format. Octopress is built on Jekyll supporting blogs.  In Javascript world, there are blacksmith, docpad, and a few more.

Out of all the contenders, for my use, I like hugo and docpad. Hugo is the simplest of the lot and extremely fast. Docpad is more versatile in supporting lot of different kind of formats, templates, and plugins.  In hugo, all that I had to do was to create a hierarchy and drop in .md files as description. Based on the folder structure, it creates the menus, content layout, and the URLs. Docpad is a bit more complex, but it is also essentially the same.

Static web with high interactivity

There is a big problem with the earlier approach. Consider the example we were giving about a document management system: what if we want to search for the right document? Or, sort the documents by date? Or, slice and dice the material? Or, draw graphs, based on the keywords?

For all these tasks, historically, we depended on web server doing all the heavy lifting. Do you want to sort the documents? Send a request to the server. Do you want to filter the documents? Send another request.

image

While this kind of interaction between the user and the web server gets the job done, it is not terribly efficient. It is a burden on the server; it increases bandwidth requirement; it feels sluggish to the user.

Thankfully, the situation changed with advancement of Javascript. Turns out when any HTML page is digested by the the browser, it creates a data structure called DOM. With Javascript, you can query and change the DOM. That means, at run time, you can do sort, filter, display, hide, animate, draw graphs – all that with the information available to the browser.

With this kind of power, we can design interactive websites without going back to the server. With this additional requirement, how we develop web pages and what technologies we use will be different.

image

See the sequence diagram titled “JS based interactive sites”. The web server sends the data, html, and javascript the first time. After that, any interaction is handled by the script itself. It can compute the new content based on the user interaction, from the existing data and make the necessary modification to the page by changing the elements of DOM.

The implications of this design option are enormous. We do not need to burden the server; we do not need to overload the network; we provide quick interaction to the customer.

The range of interactions is limitless. For instance, if we are offering an electronic catalogue of books, we can search the titles, sort by authors, filter by publishing date, and so on.

In fact, these kinds of interactions are common enough to have several mature libraries supporting them. For example, for the tasks I mentioned in the previous paragraph, I would use dataTables. If I am doing the same with millions of entries, I would use Pourover by NYTimes (which used this library for their oscar award fashion slicing and dicing web page).

Static web for collaboration

For the kind of interactivity we discussed earlier, Javascript libraries work well. If you think about it, all those web pages are providing is read-only interactivity. What if you want read/write interactivity?

For example, if you have a website with lots of content. You want to provide people a way of adding comments. Take the case of earlier Octopress itself – we may want to add commenting capability to those blog posts. How would we do that? We certainly need server side capability to do that.

Fortunately, the server-side capability need not come from us. We can easily use 3rd party server for these capabilities. For instance, we can use disqus or Facebook for comments. We can use Google analytics to track the users.

image

In fact, the 3rd party server ecosystem is so strong these days, we can develop highly interactive websites, with just static content served out of our server. You can learn what other leading web companies are using on their web pages and what services they are using from http://stackshare.io/.

For example, if you want to use payment services on your website, what kind of 3rd party service can you use? Look at the choices: http://stackshare.io/payment-services: Stripe, paypal, Braintree, Recurly, Zuora, Chargify, killbill and so many others.

How do you integrate into your website? Here is an example:

<form action="/charge" method="POST">
  <script
    src="https://checkout.stripe.com/checkout.js" class="stripe-button"
    data-key="pk_test_6pRNASCoBOKtIshFeQd4XMUh"
    data-image="/img/documentation/checkout/marketplace.png"
    data-name="Stripe.com"
    data-description="2 widgets"
    data-amount="2000">
  </script>
</form>

This action creates a checkout button. When the user checkouts, it creates a token and provides the browser. You do need to have some server side component that takes this token and charges the user – it goes beyond the usual static website I was describing. But, for most other less critically secure website, you can conduct the updates from the browser itself.

For example, take a look at how you would integrate the analytics from Google on your website:

<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-XXXX-Y', 'auto');
ga('send', 'pageview');

</script>
<!-- End Google Analytics -->

You find this code in several websites, which posts a click data to the Google server.

Admittedly, most services need server to interact with them, for security purposes. Nevertheless, the heavy lifting is done by some 3rd party web server.

Static website special case: Server less setup

A special case of static website is desktop website. Suppose you want to share a folder with your friends. Easiest way to put the folder in dropbox and share it with them. Now, suppose, you want to provide some interactivity, say, searching the files etc. What would you do? You could host the files in a website. Too much trouble. You could run a local webserver. But, that is too complex for most people. Why not run the site with file:// protocol, without requiring a server, directly opening a file in the browser?

This approach works surprisingly well. The workflow could be as easy as this:

  1. Let people, including you, place the files in the shared folder.
  2. Watch the folder on update (or, do it periodically) and run a script that generates the data file.
  3. The user can open the index.html folder, which uses the data file to create a web page.
  4. With suitable library (like datatables) the user can navigate, search and sort the files.

This is a simple poor man’s content management service. You can enhance the document authoring process to add all other kind of meta data to files so that you can make more effective slice and dice operations of the folder.

Static web for e-commerce: An exercise for you

Let us take it up one more notch. Can we design an entire e-commerce site as a static website (or, at least with minimal server components)? We want to be able to display the catalogue, let the users browse, discover, and even read/write the comments. In addition, we should let them add items to shopping cart and check them out. We may even want to add recommendations based on what you are looking at.

Now, how many items can we keep in the catalogue? Remember that images are in separate files. We only need the text data. Looking at general information, it is max 2K per item. While there is no limit to the amount of data browser can load, anecdotal evidence suggests that 256MB is a reasonable limit. So, 100,000 items can be displayed in catalogue, without problems. Remember that all this data and images can be served out of a CDN.

We can do one better. We do not have to load all the items at once. We can load parts of items, based on demand. Now, if the commerce site has different divisions, and the customer chose one of them, we only need to load that part.

If we can reduce the number of items to say 10,000 to start with, that makes it less than 20 MB, which is the size of a small video. So, it is entirely reasonable, for user experience perspective, to be able to load 20 MB for a highly interactive site.

What about other services? We can manage the cart in JavaScript. The actual checking out: payment, receipt, and communication to the backend need to be done an actual server. Anything less would make the system less secure. Anybody with knowledge of the JavaScript can easily spoof the browser – so, best not to make direct calls to the backend, from the browser that assumes any valid data from the browser. All you are doing is a providing the integration in the browser!

We can think of some more modifications. What if we design a mobile application? We only need to ship the deltas in the catalogue. After choosing the catalogue, the application can send a request to fulfillment with some additional credit card information.

Now, go ahead do the following tasks:

  1. Draw the technical architecture diagram
  2. Create a data model and generate sample data in JSON
  3. Create a set of javascript programs to support the following activities
    1. users browsing the catalogue
    2. Adding items to the cart
    3. Checking out cart (here, you might some server components – do it an a different server).
    4. Searching the catalogue
    5. Managing the cart
  4. For additional credit, do the following
    1. Cross-selling and upselling (people who bought what you looked for also bought the following; or, buy this additional item at a discount). Discuss the security implications.
    2. Develop a mobile application that caches the entire catalogue on the device. Figure out a way to keep the catalogue data synchronized on demand.

Have fun!

Dec 312014
 

Because of Facebook, I have been in constant touch with friends, acquaintances and even people that I did not meet. I am using this annual letter as a way of summarizing, introspecting, and filling in the gaps in my usual communications about technologies to friends. It is heavily slanted towards technology, not the usual intersection of business and technology.

There are three ways that I learn about technologies. One is by experimenting on my own. By actually coding, practicing, verifying hunches, validating ideas, and playing, I learn a bit. By talking to customers, sales people, engineering managers, and developers, I understand what the problems of application of technologies are. By reading books, news papers, and blogs, I try to see the interrelationships between different subjects and the technology influences in the modern world. Let me review from the annual perspective, what these three different influences taught me.

(Cloud) Container based technologies

I played quite a bit with container based technologies. Specifically, I tried various docker based tools. In fact, I setup systems on Digital Ocean that lets me create a new website, make modifications, and push to public, in less than 10 minutes. That is, using the scripts that I cobbled together (most of them are public domain, but I had to make some tweaks to suit my workflow), I can create a new docker instance, initialize with the right stack, provision reverse proxy, and push the content to that machine, install all dependencies, and start running the system.

 image

Minimal, cloud-ready OS’s

From my experiments with docker, I see that the power of virtual machine that can run any OS is not much useful. I don’t want to think in terms of OS. What I want is a stack to program in. My OS should be cloud-aware OS. It should be minimal, should work well in a hive, should support orchestration, and should be invisible. Since I am unlikely to interactively work in it, I would place a premium on programmability, and support for REST services. Of course, it needs to secure, but since it is minimal OS, I want security at different layers.

image

Based on all these requirements, I see hope for CoreOS kind of OS. I don’t know if coreos is it, but something like that — a minimal, cloud ready OS is going to succeed. Companies like Google and Facebook already use such systems for their internet scale applications.

(Server side technologies) Node.js technologies

I entered this year having learnt a bit about node.js. I have a love-hate relationship (who doesn’t?) with JavaScript. On one hand, I love its ability to treat functions as first class objects, its polymorphism, its ability to extend objects etc. Its approach to type system is dynamic, flexible, and incredibly powerful.

Yet, I find lot of things to hate about JS. It scoping is infuriating. Its support for basics of large scale programming are absent. Its ability to type check are minimal. It leaves our crucial features of programming, letting users create competing idioms.

For small to medium systems that are cobbled together by REST services, node.js still is a quick way of getting things done. And, I like npm — it is even better than CPAN. I am not a fan of reinventing the wheel with all the tools like gulp, bower etc. The proliferation of these tools is confusing, putting off a casual user with their own standard tools. (Java was the original culprit in these matters.)

image

In Node.js, the technologies I played with are:

  • Express: Of course, duh. There may be better ones there, but I needed one standard one in my arsenal. This one will do.
  • Mongo: The power of using JavaScript across all the layers is seductive. I don’t have to translate from one format to another. And, let somebody worry about performance (well, not actually, but will do for the most part).
  • Usual set of libraries, involving, parsing, slicing and dicing, and template management.

Front end JavaScript

I have been frustrated with the front end technologies. There are too many frameworks. MVC? MVCC? And, the complex edifice makes my head swim. At the end, I am not able to make sense of it all. Thankfully, I see things changing. I am able to settle down on a few choices for myself — not because they are the best (in some cases, they are), but they are good for a generalist like me.

JQuery: I still find it the best way to manage a DOM from JS. I never bothered to learn the full DOM API, and I find JQuery convenient way of managing.

Web components: Specifically, I fell in love with Polymer. Why? Think about this. HTML has a standard set of tags. Suppose you want to introduce a new tag. You need to create JavaScript that parses the new tag and manages it for you. So, your code is strewn in a few places: the JavaScript code, CSS specifications, and the HTML. It is too messy, too unmaintainable, and more importantly, difficult to mix different styles.

Enter web components. You can create new elements as first class entries in DOM. The looks of the new element are specified via CSS in there itself. The behavior through the JavaScript also goes there. You expose specific properties and interactivity. The big thing is since it is first class element in DOM (as opposed to translated to standard elements through JS), you are able to reference it from JQuery and manage it just like you would a heading.

image

Since not many browsers implemented web components, we need a Polyfill, a way of mimicking the behavior. Thanks to Polymer, now we have JavaScript code that makes it appear that the browser supports this DOM behavior of web components. This polyfill intercept every call to DOM and translates appropriately.

Summary: It is slow and buggy at the moment. In time, it will take off, creating a nice 3rd party market for web components. It almost like what COM did for Microsoft.

Assortment of libs: While I did not reach my ideal setup (where the machine can speak IKWYM – “I know what you mean: language), there are several libs that help me now. Specifically, I like the templates with inheritance like nunjucks. I also use Lodash to make life simpler. And, async.js to use the async paradigm of JavaScript.

HTML looks and feel

As an early adapter of Bootstrap, I stand vindicated in pushing it for my corporate users. Now a days, almost all development we do is responsive, built on standard template driven development. Within that, I dabbled with a few design trends because I found them interesting:

  • Parallax Effect: You see pages where the background images roll slower than the text? It gives a 3D layering effect. It is particularly effective in creating narrative stories. When the images are also responsive, this 3D effect can make the web pages come alive. To take a look at some examples, see: http://www.creativebloq.com/web-design/parallax-scrolling-1131762

  • Interactive HTML pages: Imagine you are telling a story, where by changing some choices, the story changes. For examples, tabs change the content. But, imagine creating new content based on the user input, not merely showing and displaying the content. For instance, once we know the name of the reader, age and other details, it is easy to change the text to incorporate those elements. Or, if we know what they need, we can directly address in a sales dialog. While I did not carry out this idea to completion, I satisfied myself which the technology and the framework to do this interactive narrative sales tool. Naturally, this framework has a little bit of JS magic.

  • Auto generation of web pages: As I was writing text, converting the text to HTML and a suitable web page became an obsession with me. I finally settled down to using combination of md5, bootstrap, and yaml properties to generate a good looking web page to share with people.

If you are interested, please see these two blog posts, from yester years:

  1. http://www.kanneganti.com/technical/engineers-guide-to-understanding-web-design/
  2. http://www.kanneganti.com/technical/html5-design-knowledge-for-application-developers/

Static web apps

As I started seeing the advances in the web front ends, I see the possibilities of static web site. For instance, it is easy to create and run a static e-commerce application, with 10K or so SKU’s without any trouble. We can even have recommendation engine, shopping cart, various payment methods — all these thanks to web services and HTML5.

image

The following are the technologies that I found useful in the static websites.

  • markdown for html generation: For generic content, markdown is easy to author format. In fact, we can even author books in this format.
  • asciidoc for book authoring: For larger format HTML with more whizbangs, asciidoc is what I tried.
  • docpad for static website generation
  • pandoc for format conversions

For the in-page manipulation of large number of objects, I find the following very useful:

  • pourover: The amazing slice and dice web sites for displaying large tables is done by pourover library from NY Times. I have high hopes for it. I think there are lot of innovative uses for this library, with its performance, and ability to cache the results.

One of my favorite problems is to develop a web applications, without any db in the backend, a pure static application that acts as a library interface. We can search, slice and dice the selection using various criteria. For instance, latest document about a particular technology, written by author XXX, related to YYY document.

Mobile & Hybrid application development

I have for a long while, bet on hybrid application development, instead of native application. Now, I concede that on the higher end market, native apps have an edge that is unlikely to be equaled by hybrid applications. Still, in the hands of an average developer, hybrid platforms may be better. They are difficult to get wrong, for simple applications.

image

This year, I was hoping to do native application development, but never came around to it. With polymer being not yet completely ready, I dabbled very little with Angular based framework called Ionic. It was OK for the most part.

Still, for simple pattern based development, I hold lot of hope in Yeoman. For corporate contexts, one can develop various scaffoldings and tools in Yeoman generator framework. That leads to compliant applications that share the standard look and feel without expensive coding.

Languages

In my mind, right now, there are three kinds of languages: ones that run on JVM — that includes scala, Java etc. Ones that translate to Javascript: these include Typescript, Coffeescript etc. And, the rest, like Go etc. Innovation in other languages has slowed down.

a1

Despite that, the three languages I dabbled this year are: Scala for big data applications, specifically for spark; Python, again for data applications, specifically statistical processing, and Javascript, as I mentioned earlier. I liked typescript, especially, since it has support from Visual studio. I started on R, but did not proceed much further.  Another language I played with a bit is Go, in the context of containers and deployments.

Data and databases

This year, I gave a 3 hr lecture in Singapore on bigdata, in the context of internet of things. I should have written it up in a document. The focus of that talk is what are the different areas of interest in big data are and what technologies, companies, and startups are playing in those areas.

image

This holidays, I experimented with Aerospike, a distributed KV database developed by my friend Srini’s company. Whatever little I tried, I loved it. It is easy to use, simple to install, and fast to boot. According to their numbers, it costs only $10 per hour to do 1 million reads per second on google compute platform. I will replicate and see how it compares against other databases like Redis and Cassandra that I am familiar with.

On the statistics front, I familiarized with basics of statistics, which is always handy. I focused on http://www.greenteapress.com/thinkbayes/thinkbayes.pdf to learn more. I followed Quora to learn about the subjects in Coursera. I wanted to get to machine learning, but that will have to wait for 2015.

On particular aspect of big data and analytics that fascinates me visualization. I played with D3 — it was of course the basis of most of the visualization advances that we see these days (http://bost.ocks.org/mike/). I am on the lookout for other toolkits Rickshaw. I will keep following it to see the new upcoming advances to make it more main stream.

Other computing tools

I find myself using the following tools:

  1. Atom for editing text files.
  2. Brackets.io for editing html files, and for live preview of the static web pages.
  3. VMWare workstation for my virtual machine needs.
  4. Markdown for note taking.
  5. Developer tools from Chrome for web development.
  6. Visual studio for web development
  7. Digital Ocean as my playground of coding activities
  8. OVH for any large size applications and servers

Customer conversations

Since most these conversations have some proprietary content, I cannot give full details here. In general, the focus in on innovation and how corporations innovate in the context of established businesses. Typically, it is a mix of technology, processes and organizational structure transformations to change the way businesses are run. I will need to talk about in byte size postings some other time.

Wish you a happy 2015! May your year be full of exciting discovery! See you in 2015!

Aug 222011
 

It is obvious to most people that, at heart, I am a programmer. I am not a good artist. My wife wouldn’t trust me to pick colors for our bath room. She wouldn’t even let me pick the tiles! But, obviously, I am qualified to make judgments about UI design Smile.

image

Deprecations aside, I worked hard to compensate for my innate weaknesses: I spend long hours on the internet. Whenever I see a website, I view the source and then understand how they did each page. I try to even guess what kind of software they used in the backend. I regularly check out standard sites. I took college level courses in history of art.  In my misguided youth, I even designed fonts. I am writing al this preamble only so that you don’t think that I am ranting.

Long back (in the internet time, it means just 5 years back), we used to get user interface designers. We expected little from them because we didn’t know much. All we told them was to make it look pretty. What we used to get is always a windows interface tacked on to the web:

image

Essentially, you have a menu bar, and then the menu bar lets users perform activities on the data in the screen. The menu can be on the side, but that is still the same principle. Even if you have multiple menus, it is somewhat same.

What is the problem with this? Nothing, if you are addressing the same audience. But, most applications on the web are not like windows applications. They are for different purpose, for different audience, to be used in different context. For instance, they may be for casual users who do not know all these menu’s are about. Or, they may be for doing a series of standard steps. Just like how, even in windows, you have different kind of UI’s for specific tasks (like say, a wizard), you do need to adapt to the activity and the users.

If we want something better than this monstrosity, what do we need? Since sky is the limit, let me stipulate the following:

  1. We are not developing a world class system where UI is the main innovation. We know that the difference between facebook and myspace is say 50(?) billion dollars and better user interface. If you have such great ambitions, what I am going to say may not work for you.
  2. We also want the best UI that our limited budget is going to buy. Surely, it is the functionality we are selling. At the same time good UI make the functionality actually usable.
  3. We are building enterprise applications with only limited number of users. We are not building a website for the millions.

You may think it is restrictive – but wait till you see remedy interface. I wish they spent all of $500 on the interface!

What I expect the UI developers to do

With those goals, if I were to design a web applications, this is what I would want my UI people (or UX people) to do:

To be in between users and developers

I want my UI people to know the users well enough to disregard UI requirements and create a cohesive and coherent interface. UI design is not about focus groups and satisfying different users. It is about understanding what users really want to do and provide a consistent way to do it.

Similarly, they should understand enough about developers to see map the requirements to familiar patterns. Most developers don’t develop well, if you ask them to do it from scratch. They can imitate well; they can reconstruct reasonably well. UI developers should be able to map the user requirements to familiar patterns in the developer toolbox.

Adapt Industry best practices

Either we lead or we follow. Looking at the constraints that I laid out earlier, I say we should follow the leaders. There is nothing intuitive about any design (except some – like pulling a loose cord, or pushing a stone – blame it on our nomadic hunting days). Most of the complex paradigms are learnt over years of experience from using the systems.

image

And, innovation over those complex interactions is somebody else’s job. Your job is merely understanding the right innovation and using them in your system. In fact, I have seen that any innovation fail miserably because of lack of talent, support, or budget. So, stick to what worked outside.

Integrate the standard paradigms

Most enterprise applications are about data entry and queries. Workers add information and managers review information, approve applications, or get reports. There are lot of variations in the usage paradigms: workflow, process flow, content management, user interaction, social networking, CRM, helpdesk, self-service, e-commerce etc.

From http://ui-patterns.com/

Most modern applications are a combination of all of these. A good UI person understands the nature of the application to get the UI representation of these standard paradigms and map the user requests to the design. If not, what we get is a bunch of pages, that ignores the connections and neglects the possibilities of achieving a coherent vision. Worse still, it will not leverage the established UI patterns and therefore ends up being too costly.

Manage the costs of technology

I recall an incident that a friend of mine told me. The incident happened in the hype of Java applets age. The client wanted the comments field to support variable width fonts and some limited formatting capabilities. The UI person, who was working on the wireframes put that as a part of the design. Unfortunately, the design was not supported in HTML in those days (no/low css; low adoption of JS). So, the developers were forced to use Java applet. That complicated development, testing, and deployment. When the final ballooning of the cost was shown to be more than $200K, the client was dumbstruck. “Why didn’t you tell me that it was going to cost so much – it was not a big deal!” was his last plaintive cry before being wheeled into emergency room.

image

So, I want my UI people to know the standard Javascript libraries. I want them to know the what plugins exist, what are the costs of doing different interactions. I want them to know some security repercussions of the designs as well. And, with that understanding I would like them to design the user interactions.

Work with templating systems

I often find in the following difficult situation. I get a bunch of screens (HTML +CSS). I do not know the relationship between the pages. Not the navigation scheme, but the relationships. As a programmer, I am used to seeing pages as a hierarchy. Like, say, the results page is just like any other page except the results are in the middle of the page.

(Repetition is good, if we can identify the hierarchy and differences)

In short, if we are able to create a hierarchy of pages, each child specializing the parent page by adding more element, I can quickly prototype and build a system that can be changed at will. If not, each page will get programmed and changing the look and feel of large number of pages depends on the programmer skill.

Unfortunately, here we get into a snag often. The tools the UI people use do support this kind templating system (Dreamweaver and so on), but they do not or cannot expose that to the developers because the developers use different kinds of templating systems. I cannot offer a universal solution – all I say is, the UI people should be able to work it out with the developers.

What I look for in the UI designers

Now that I specified what I want them to do, I will tell you what I look for in the interview:

Knowledge of CSS

Most UI developers are quite good at this. If we want low cost solution though, your best bet is to go with something standard like using Jquery UI. If we are using Jquery UI, it comes with its own themes. Any average UI person should be able to develop the themes to meet the client requirements. The advantages are many fold: you can use lot of plugins as is. You can reuse developer skills as well. [Here is a simple introduction.]

image

Knowledge of Javascript

In the modern days, any reasonable UI requires a Javascript person on the team. Even the UI people should be familiar with the standard libraries, what they can do, which are the ones that we decided to go with and how to use them to achieve the interactions we need. For finer customizations, we may need a full-fledged JS developer, but most of the basic ones can be done by UI developers.

image

Color schemes

I know that this is supposed to be the forte of the UI developers. I have often been disappointed with the UI team’s failure to develop a good color scheme. If you are forced to do it, just look at these two resources: color wheel and kuler.

image

Icons

A simple good set of icons can spice up the design and get user attention easily. Here is where we can create a unique branding, play on the color scheme, and provide good visual relief. If the UI designer cannot design the icons, at least he/she can buy them or get them from public domain. Ability to incorporate the icons in the web design for consistency and usability is something I look for in a UI developer.

image

On the issue of stock images: Most corporate applications get spoilt by usage of stock images. How many times have you seen a bunch of white people in suits, with couple of minorities thrown in to indicate seriousness of the site? I agree that you can create a good ambience and set the tone for the site by the stock photos – but that is rare enough that I lost faith. In any case, if you want good stock images free, you can always go to this site. Summary: Don’t expect too much in the way of stock photos from your UI designer.

Fonts

A few words: CSS; standard set of fonts; good readability; ability to use web fonts. And, of course, use them as a theme for using Jquery UI or some such system. If your UI engineer can do that, that would be good enough.

PS: If you are a developer and think that your designer is producing that is way too difficult to develop for, or not doing a good enough job, you can do a few simple things:

  1. Use Jquery UI for your design
  2. For your screen layout, check out the following: http://www.openwebdesign.org/. For instance, I found this http://www.openwebdesign.org/design/3499/multiflex32/ design – a reasonable, if a bit old fashioned one.
  3. For stock images, check out: http://sxc.hu
  4. For your icons: There are several open source ones. While I like the ones from the nounproject, they are not in color.
  5. For your fonts: check out http://www.google.com/webfonts
  6. For your UI paradigms, you are on your own – you need to understand the user needs and what your application is about!