Oct 212013

This post has nothing to do with whether “obamacare”  is good or bad. It is only about the discussion of the technology stack and the details of it.

At $634M, it is one of the costlier government projects. At its launch, it ended up failing for several users. Even now, they estimate 5M lines of code change to fix the system. What is going on?

The problem, viewed from one angle is simple. Let the users discover and onboard to a particular plan. And, it should cater to large number of users. Since the users do not have other options, you can dictate terms to users (you could say that they need to download a specific version of browser to work with it). Looks easy enough.

On the other hand, the problem is complex. There are any number of plans. There are several different exchanges. The eligibility criteria is complex. There are different agencies, databases, vendors involved. And, the integration is bound to be complex. So over all, it is, well, complex.

To top it off, it is overseen by government agency. These people are good at sticking to rules, procurement, checklists etc. If they check for the wrong things, the site meets all the requirements, and yet, fail.

Tech stack details:

The tech stack is modern. They rely on JS on the browser.

Performance issues

Summary: They are doing lot of rookie mistakes in optimizing the page for performance. With minimal effort, they can take care of most of the issues.

  1. They seem to use 62 JS files in each page. They need to use fewer files and minified as well to reduce the round trip. With 62 files, and that too, without expires headers, we are looking at 15 round trips and that means around 5 seconds of loading time itself (assuming .3 sec for round trip and processing).
  2. The page is heavy! The JavaScript is 2MB and the css is .5 MB and images are .25 MB. So, over all, the site needs to download 2.75MB just to start working.
  3. For the returning user, the situation is only marginally better. They still need to make 85 round trips (that is the number of components); but they only need to download .5 MB.

If experienced folks developed this site, they can reduce the round trip time to less than 1 second (5 fold improvement), easily.

Code quality issues

First the tech stack details.

  1. Bootstrap
  2. JQuery
  3. Backbone
  4. Underscore
  5. JSON
  6. JQuery UI (Why?)
  7. Parts of Scirptaculous

Their stack is not to blame. They are making use of API’s heavily. They use bootstrap (version 2.3), Jquery, backbone, underscore and JSON. I think backbone is too complex a technology (I am partial to Angular, or lot of other modern JS technologies), and the rest are simple enough. In the hands of novice developers Backbone can get very convoluted. In fact, the same can be said off JS. 

Let us take look at code quality (what we can see in JS files):

  1. Backbone is complex for these kind of apps. That too for average developers, BB tends to be difficult to use.
  2. Checkout the file: https://www.healthcare.gov/marketplace/global/en_US/registration.js – to understand how the code is laid out. They are not doing template driven development or metadata driven development. This is too much of hard-coded stuff. And, look for “lorem ipsum” too, while you are at it (that shows poor test coverage, or unnecessary code). (this file may be auto generated…).
  3. Use of too many technologies: Shows sub-contracting and no architectural oversight. For instance, if you are using Bootstrap, might as well stick to it, instead of getting JQuery UI stuff. Also, lot of JS files like Carousel etc are built in Bootstrap – why have separate ones any more?
  4. Lot of code in JS seems to have been generated by MDA – that may account for some of the mess. Check out the MDA tool: http://www.andromda.org/index.html

At this point, I don’t have much information: The github pages seem to have some code, but that may not be the one used here. The github uses static html generator (a good strategy) – but that is not what the current website has (Github code seems have been removed now).

Overall, it is looks like high concept, reasonable execution, bad integration, and terrible architectural and design oversight.

I will return with more, when I get some time to analyze the site.

 Posted by at 9:58 am

  2 Responses to “Healthcare.gov– a tech discussion”

  1. The overall system is architected along the typical lines of an EAI – front-end app implemented by the government and the exchanges (with the tech stack you covered here), an integration app (my team’s implementation) and the back end legacy insurance and payment apps (that are, in some cases, running 15-year old technology). Kinda like MVC.

    Data from the front end hasn’t even begun to be passed on to the integration app yet, because of the scalability issues at the front-end. Millions of enrollment requests have piled up there, unable to swim through the molasses-like quagmire that is the front-end system.

    You’re right. The code in github isn’t the one on the exchanges. In fact, each state’s exchange uses a different code base. You’re also right on the lack of testing. Incredibly, the CMS (Center for Medcaid Services) began creating and sending out the first 2-4 test cases a couple of weeks before go-live (October 01).

  2. Overall, this is positive. The healthcare site has instantly become the best sales tool for: lean,

    agile, lightweight stacks and flat subcontracting models.

    healthcare.gov’s failures are not uncommon—they’re just exceptionally high-profile

    Useful links:

    * The subcontracting breakdown (via reddit)


    * UX critique


 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>