Jan 092013
 

A famous saying goes that you hire people who are “smart” and “get things done”. I cannot tell you if people can get things done from their innate capabilities or intelligence. But, over time, I realized that most intelligent people that I come across have the similar qualities. I struggled to characterize them until I came across this list in the book “Gödel, Escher, Bach: An eternal golden braid”. Quoting verbatim, here they are:

  1. to respond to situations very flexibly;
  2. to take advantage of fortuitous circumstances;
  3. to make sense out of ambiguous or contradictory messages;
  4. to recognize the relative importance of different elements of a situation;
  5. to find similarities between situations despite differences which may separate them;
  6. to draw distinctions between situations despite similarities which may link them;
  7. to synthesize new concepts by taking old concepts and putting them together in new ways;
  8. to come up with ideas which are novel.

 

I must say that this list is very satisfying. After coming across this list I started using it it my daily life. Am I exhibiting these qualities? Am I evaluating people based on these qualities? Do they correlate with other ways I evaluate the effectiveness of people?

I use this list in multiple ways. I use it to evaluate and improve myself. I use it to evaluate people that potentially work with me. I use it in my activities as an architect, as a technical evangelist, as a coder, as a pre-sales principal, and as a strategist.

image
(©: New Yorker)

To respond to situations very flexibly

I find several people who excelled at school fall short in life later on. People prepare well for well-set idealized scenarios. When the situation differs from what they learnt, they cannot respond flexibly.

For instance, let us see how it works in presales situation. You are prepared for a standard scenario that a customer might face – say, how automation of processes helps to save the money. You go to a customer for whom automation costs lot more than actually the savings that get accrued. If you forget why you were proposing the solution (to save costs), you end up needlessly pushing automation, when all they want is effective reduction of process execution.

In my experience, the flexibility demonstrates the ability to understand the big picture and context of the problem that only comes with a deep understanding of the subject or solution you are trying to present. I find that the following helps me greatly:

  • Understand the history: History offers a way of understanding subjects and how they come about. If we don’t understand why people are using certain tools or methods, we may not be able to offer a better way. For instance, the why of operating systems offers why we treated files differently from the network connections. Then, we might start thinking why not treat them the same way. We start seeing parallels from history and apply those as the situation demands.
  • Have a core set of logic tools: While this rule may appear generic, I find that most amount of flexibility in our thinking comes from understanding what is essential and what is not. My set of tools are from model theory – in particular higher order models. I start seeing things from the perspective of completeness and consistency, even when the situation presents in a new guise.
To take advantage of fortuitous circumstances

I am not sure if I agree that this is an essential quality of intelligence. But, still, most successful people have the ability to see opportunities that others don’t. I think it takes courage, self-confidence, desire, and deep sense of conviction all of which are not necessarily attributes of intelligence.

I am also fairly sure that I am not so good in this area. I do not know how one can “improve” here. I suppose this is some innate capability people have, either by nature or nurture.

To make sense out of ambiguous or contradictory messages

I read science fiction stories where species communicate with precision, the exact emotion or facts. I read about conlangs (constructed languages) that are incredibly precise. But, in reality, we deal with lot of ambiguity, imprecision, and sometimes deliberate obfuscation.

Most of us deal with this ambiguity daily. We find an organization pursuing unclear and multiple conflicting paths. We find people making contradictory statements. We may not see consistency in action and word.

It may be fodder for sitcom or science fiction humor – a fish out of water story of foreigners, extra-terrestrials not possessing the verbal intelligence of natives. The consequences for an organization are more disastrous: infighting, multiple directions, and inaction. It forces leaders to constantly explaining people what to do.

While it is easy to dismiss that it is innate capability, I think there is a way to develop it well, in particular, the following skills, which are strongly correlated to this capability:

  • Reading widely helps: In particular, I think reading good fiction, poetry, and even nonfiction helps us understand the ambiguity and imprecision in language. Reading classics has an advantage as they were widely interpreted from different perspectives.
  • Writing well helps: Right kind of writing can help cut through ambiguity. When I don’t understand a subject well, I try writing, just as a way of clarifying it for myself.
  • Understanding of logic helps: In particular, I go back to my first discipline, model theory and higher order logic to understand the model in terms of consistency and completeness.

image
(©: New Yorker)

To recognize the relative importance of different elements of a situation

One of the big milestones in growing up is understanding cause and effect. That feedback loop helps us to know the consequence of our actions. The same knowledge, when applied gives a way to assessing the importance of different elements of a situation.

Fortunately, this is easy enough to deal with. At the risk of generalizing and simplifying, the best way to deal with is through metrics and goals. If you know the end goal, and the impact of the tasks on the ability to reach the end goal, you know how to prioritize. For a full discussion of the topic, see: http://bit.ly/J8b3BW.

To find similarities between situations despite differences which may separate them

Abstraction of the core essential elements from different situations is a key human trait. Right kind of abstractions can help people understand the core situations. For instance, if we want to solve a problem, using the right abstraction, we can bring tools and processes from similar solutions.

Even in presales, customers would love to know how you solved similar problems for others. Everybody knows that exact problem is difficult to find. Finding a useful abstraction is key to creating a similar set of problems that can help solve the problem at hand effectively.

Purity
(©: xkcd.com)

To draw distinctions between situations despite similarities which may link them

The flip side of abstraction is reification or concretization. Every problem is different. Only when we understand the differences between the two different problems, we can understand what kind of customization we need to do for this problem. It may be a simple change to a tactic; for being aware of such need is key to look for such change.

For instance most transplanted solutions from a different culture do not work in another culture. The cultural context, which makes a situation vastly different requires change in tactics and strategy. Without that sensitivity, people misunderstand the situation. (“Iraq is just like Germany after the second world war – all it needs is democracy” – see how that works).
Space-time is like some simple and familiar system which is both intuitively understandable and precisely analogous, and if I were Richard Feynman I’d be able to come up with it.
(© xkcd.com)

To synthesize new concepts by taking old concepts and putting them together in new ways

There is a saying: “There is nothing new under the sun”. But, then, we see new creations from human ingenuity come every year. Most of them are novel applications of existing ideas. In my own field of programming languages, I see the ideas from 80’s becoming popular now. When I see CPS (Continuation Passing Style) in Javascript, I am home work assignments from my student days.

Creation of new concepts from old takes several pieces of puzzle:

  • Good understanding of history: Why this problem is important, what techniques were tried, what was the genesis for the final solution – all of these are important when we need to put together a new solution based on the old concepts.
  • Good understanding of the changes in situations: For instance, when we were solving main memory databases in 90’s, we had maximum of 4GB in general machines. These days, I have 64GB on my desktop machine. Naturally, the assumptions have changed, opening up the solutions that were not considered before. For example, changing economics of printing changed the way books are published.
  • Good understanding of the old concepts: Unfortunately, learning, especially in computers has become more and more understanding the details instead of concepts. Details will tie us down to an existing way of thinking. Understanding the core concepts (why, how, and what) lets us put them together in new ways.

Copied from interwebs

To come up with ideas which are novel.

I suppose, this is what classically considered intelligence and genius. I do not have any suggestions on how one goes about improving in this area. I find that I am a pastiche kind of person – I take a concept and apply in a different area. Personally, I do not think I have any ideas that can be called truly novel.

Concluding remarks

Looking at the core traits of intelligence, I can only conclude that some of them are hard to acquire. I do not know how one can train oneself to come up with new ideas. But, as for other traits, there is hope that we can practice deliberately to hone those skills. Fortunately, the world is large; there are lot of ideas coming from different places. All you need is to pick a set of tools that are good enough; and master them; and apply them consistently to learn these skills.

Happy practicing!

 Posted by at 10:00 pm
Nov 052012
 

An interesting article appeared yesterday in NYTimes titled “A Capitalist’s Dilemma, Whoever Wins on Tuesday” . While the title appears to be topical about elections, it is broader and more applicable to the modern times we live it. In fact, it resonates well with what I have been saying all along.

Long time readers of my blog know that I have been a proponent of innovation along three layers: Systems of record, systems of change, and systems of innovation. The first group focuses on efficiencies, the second one of better outcomes, and the third one on disruptive changes. Clayton Christensen (He is most famous for his book, innovators dilemma)  seems to say something similar, but from economic perspective.

In his worldview, as a finance professor, he sees three uses for investments:

image

In fact, this thought process is not new. In the classical Marxism, the concept of inherent contradiction in capitalism alludes to something similar. As capital gets accumulated, it leads to efficiencies, which reduces the need for people causing depression in demand and so on. Modern economists (Example: Galbraith) focused on how to break this depression cycle by creating disruptive industries. Fortunately, US managed to come out of depression by empowering innovations.

The current situation is this: There is lot of capital available. In fact, tons of it. The reason why is not being used is neither the regulation (which is what the right wing economists want us to believe), nor the lack of demand (which is actually true – but simulating demand for a long term cannot be a short term measure). [I may be out of my depth as an economist here – I am only rationalizing and could be entirely wrong, but it makes for an interesting narrative.]

What is the problem is systemic issue.  See how we measure the ROI similarly in all industries. In India, the real-estate ROI has sucked away any investment, which itself might have created a positive effect for real-estate. Instead, it became a short-term “slash and burn” operation that extracted ROI from real-estate at the cost of empowering innovations.

According to Clayton Christensen, the problem here is the way we have been measuring the financial returns: ROCE (Return on capital employed), RONE (Return on net assets), and IRR (Internal rate of return). They can reduce the investment and still make good ratio. Or, invest only in quick wins and get good metrics.

So, what is the prescription? According to the professor:

  1. Deal with the abundance of capital – by investing in the right skills.
  2. Change the metrics – so that we use the capital for the right tasks
  3. Change the capital gains taxes to promote the right investments
  4. Change the politics to focus on the empowering innovations.

Of course, I can see the following criticism:

  1. The right wing people will see it as a way for the government meddling in picking winners and choosers and shaping the behavior.
  2. The left wing people will see it as a violation of social contract which is to focus on helping the people in need, the poor and the elderly.

Recently, there have been growing calls for investment in infrastructure, however muted they may be. Let us see how it goes after these elections.

 Posted by at 11:18 am
Oct 252012
 

There was an interesting thread on reddit about money. It is not about investing or the moral values of money, but the fundamental concepts behind money. If you have not read it, I urge you to go and read it. I will wait.

Over the weekend, I read the book “Ascent of Money” by Niall Ferguson. As a Scot, he has a unique historical perspective, that comes from the local history, including Adam Smith. As a person from the erstwhile empire, he understands the role of capital in creating the empire. I did not like the book much, but still would recommend it for some amusing anecdotes that might have to be complemented with some other serious reading.

When I see lot of main stream news (http://dealbook.nytimes.com/2012/10/22/in-london-nimble-start-ups-offer-alternatives-to-stodgy-banks/) I think there is a chance that banking is ready to be transformed. For a long time, banking has been impervious to changes. Sure, there have been periodic busts and booms (remember S&L? Home mortgage collapse?), but the same banks have come up again, with very minor changes in the way business was done. This is entirely unlike other industries, which have gone through rapid changes (when was the last time you went to a travel agent? Or, referred to yellow pages?).

Of course, there have been some changes. Rise of discount brokerage, paypal as a payment mechanism are a few of those changes.

But, that all seems to change. While there are no fundamental drivers, the momentum of change seems to have picked up. The three trends I see are:

  1. Make it cheaper
  2. Make it convenient
  3. Disrupt fundamentally
Making it cheaper

Lot of banking services are costly. For example, if I have to wire some money, it costs $35. If I want to send an overnight check, it costs me $20. That is in the highly competitive markets like US, let alone places like Australia.

Of course, they pay me 0.5% interest for the money I keep with them. If I borrow, they lend at 5% or even more.

Any such highly inefficient market lends itself to disruption. Unfortunately, two things are helping financial institutions keep their edge: one is regulation. For good reasons, it is very difficult to start a bank or an insurance company. The second is the large amount of capital to start an institution. It takes credibility, history, and deep pockets to start one.

Let us see how this market is being changed, through some examples:

  1. Dwolla.com: Payment network disruption. What it does: It facilitates person to person or person to small bank payments at very reasonable cost. It still uses ACH (I haven’t understood the details, yet). It costs just 25c to receive any amount of money. Sending is always free (as is receiving under $10).
    For more details on how it is using ACH, see this:  http://www.finovate.com/spring12vid/dwolla.html
  2. Lending club; Prosper: They make P2P lending possible. Compared to banks, it is riskier, but gets you around 10% or so rate. Historically, banks too had similar risks which were addressed by FDIC. I suppose someday, this kind of lending will be well graded, bonded, insured and all that.
  3. Gocardless: London-based company that allows small businesses to set up monthly payments to suppliers at a fraction of the cost that banks charge.
  4. Wonga.com: Short term loans (think serving the bottom of the pyramid or the long tail).
  5. TransferWise:  acts as an intermediary for the money , in particular into European currencies.

If you notice the trends, making it cheaper does two things:

  1. Identify and reduce the market place inefficiencies: The inefficiencies were not attacked before for multiple reasons. May be it was the long end of tail; may be the consumers were not ready; may be the capital was not available. But, now, the situation seems to have changed for addressing the inefficiencies.
  2. Remove the middlemen and provide only needed services: On the face of it, it looks bad. Are we getting fewer services? But, then, lot of services are as useless as a fax machine. Why do we need lot of investment in the local branches? Why do we need high priced advisors, when most research is available free?

That is how lot of companies are changing the business processes and practices to alter the banking services.

Making it easier

The other class of changes in banking services are not so fundamental; yet they impact lot more number of users. They do that by combining disparate processes, tools, opportunities (a horizontal integration) to make it convenient for users. For example, what if you can deposit a check from you smartphone? Of course, it is not a fundamental change, but still very convenient. Now, multiply those kind of conveniences several folds and you start seeing lot of changes.

Some of the examples include:

  1. Stripe.com: Credit card payment disruption. Here, this site makes accepting payments via credit cards very simple over the web. It is not like Dwolla (which sets up its own ACH etc…). It makes it merely convenient for programmers.
  2. Square: Location based payments.
  3. Google wallet: Coupons + google services + payments.
  4. www.simple.com: A new kind of bank, where they will offer the experience, and keep the money at a traditional bank of their choice. So, they are creating a new value-added intermediary.

As you can see, lot of this ease of use is addressed through adding new kind of technology. It often adds a new intermediary, but it does so, by cutting away at the traditional players.

Disrupting fundamentally

It is in this context, I am referring to Niall Ferguson’s book. The history of money is replete with examples of places where different kinds of money has been created. While I am looking to see how the monopoly of the state on money can be broken, there are interesting trends:

1. Bitcoin: A fundamental way that money is created and used as a currency. Where there have been some problems, the idea is refreshingly new and takes advantage of the concept of leverage, currency, and notes.

2. <???>: I see lot of ways bypassing money fundamentally, but none of them seem to be big enough. Please leave any observations in the comments.

As a famous net.person famously said, Software is eating the world. I hope that it will eat the financial institutions. And, we should understand it so that we can participate in it.

What does it mean to us, software developers, architects, and managers:

  1. We should understand the ways the industry is being disrupted.
  2. We should create propositions around how these institutions can address these threats. Easiest is to focus on the category “making it easier”. On “making it cheaper”, we do not have the mandate to propose new business offerings.
  3. We should create some POV papers so that we can start having right conversations with the customers.
 Posted by at 12:35 pm
Jul 102012
 

When I joined computer science in 1982, I asked a senior of mine what computers do (they were too novel at that time). He looked at me pitifully and told me that they sort records. Eventually, by the time I graduated out of IIT Madras, I learnt about compilers, operating systems, databases, programming languages and a lot of other hard-core computer science subjects. And, I thought that’s what people use computers for. And, once I entered the real world, I knew my friend was much closer to reality than what I learnt in school.

After working with computers for 20 years, when I landed into IT accidentally, it fascinated me with infinite possibilities. In a sense, IT is still figuring out what to do with computers. We can say that originally, companies automated exactly what they were doing with computers. Only recently they are figuring out disruptive models of computers – perhaps as the use of computers in social and mobile settings is becoming ubiquitous.

I have been trying to understand this evolution of IT and how to explain it well to others. I have been talking to several people from various industries to get this perspective. Hopefully, you, the reader, appreciate this perspective.

What IT wants

What IT wants is simple: it wants to support running the business, It does so by using systems. It runs the systems on computers. These computers are hosted in data centers and user desktops. It also may run the communication systems and other facilities.

In the beginning, it was not clear what this support meant. Obviously, a business does a lot of things. It is not even clear that there are similarities between different businesses, right? So, what does it mean supporting the business?

Turns out that businesses are not all that dissimilar. Thanks to the rise of the modern enterprise, there are standard parts of a business – HR, supply chain, inventory management, finance, customer relationship management etc. In fact, these “parts of business” are so standard that people learn these subjects in college during their MBA.

Now, can IT do all these things? If it can automate, what do you need people for? Actually, in the beginning the range of problems that IT was solving were few. For concreteness sake, let us consider the insurance industry.

image

[If I made mistake in the actual details, please overlook.] The gist is that the state of the art permitted only some portions of the business operations to be automated. More over, some of the automatable tasks may not be automated because IT did not know that they could be, or the problems are not modeled that way.

In the beginning: Rise of ERPs

In the late 80’s and the early 90’s, there was lot of hype around object oriented analysis and development. People had access to all the new fangled technologies: data bases, compilers, cheap computing power, and networks. Add to this heady mix, the new methodology of OO. It promised that we can uncover the “What” part of IT: “What needs to be built”?

By mid-90’s dissolution set in. Most projects stalled or miserably failed. IT departments realized that they cannot build software, merely because they have the tools. Just then, ERP became very popular. Here is what they offered (For instance SAP):

  1. Standard programs that suit most needs
  2. Standard processes for most of the common tasks in different industries
  3. Common data models for most industries

Companies started signing up aggressively. They reasoned, rightly, that these ERP solutions solve many problems in their IT: They know what to automate. They know how to automate it. Standardization provided them all the efficiencies they ever wanted. SAP went from industry to industry, functional area to functional area and offered a reasonable model to support business operations.

image

That is, there is a comprehensive platform to run businesses using a set of modules or components from a vendor. That is obviously a big step in an IT department, in fulfilling their mission.

With the internet: Integration and differentiation

An interesting thing happened along the way in the late 90’s: internet. It suddenly added a few more expectations on the IT such as e-commerce, intranet, internet based customer programs etc. Unlike the standard HR module or finance module, these programs did not have standard models. The scope, the operations, the execution – none of this was standardized.

What it meant that it was difficult to implement in ERP systems. Remember that these ERP systems create standard models (data, process, application) for users. With internet evolving at breakneck speed, companies scrambled to implement in whichever technologies they could: Perl, php, C, java, or whatever.

Why were they implementing these new systems? Couldn’t they have done in ERP? The answer is complex: you could say that ERP systems did not have the technical capabilities that the new applications demanded. That was true, but only in the beginning. To understand main reason see the larger context:

image

First a little terminology: systems of record are the ones that are standard in any business. The systems of differentiation are the ones that distinguish this business from other businesses. In fact, they are sufficiently different that ERP’s have not made it a part of their standard offerings.

As you can see that there are two ways that these systems can be built:

  1. Customize the ERP systems: Naturally, ERP vendors are smart enough to offer various kind of mechanisms to customize their systems. They offer ways to customize the data models. For instance, if you want to add one more field to track your inventory, you can easily do that customization. You can add to the business logic using their programming systems (for instance, you can use ABAP to customize SAP).

    Unfortunately, while it is clear how to customize the ERP systems, they have two problems:

    • The cost of customization is high: When you hear of jokes about ERP systems being “corporate crack” you understand the high level of frustration about never-ending projects about ERP customization.
    • Upgrade costs become almost as much as implementation costs: Most companies who implemented ERP in the mid 90’s started realizing the hidden costs of customization in mid 2000’s. The cost of upgrades have several components: understanding and documenting the customizations, upgrading the customizations, increased costs of inducting new developers and analysts, not being able to use off-the-shelf components etc.
  2. Build new systems: Building new systems is easy – in fact, most developers prefer to do that. In the beginning days of the internet, thanks to rapid development frameworks offered by the scripting languages and other systems, people developed lot of applications. However, they fall into the following trap:
    1. Not integrating well into the ERP eco-system: For example, if you added a training system like Moodle into your application soup, you have to address how that system integrates with your HR system (which may be in ERP). You have to make it both use as well as contribute corporate data.
    2. Not coordinating well with other custom systems: To take the same example, your training system may have to be a part of the process with other systems as well as ERP. For instance, if you designate a course on industrial safety as mandatory, you have to use moodle to enforce that mandate.
    3. Lack of governance and control: Suppose some other department decides to implement a blogging system (say Drupal) that also has capabilities to run training programs. If you don’t have good governance, your training programs end up fragmented which make the coordination and integration difficult.

What it points out is that while custom applications may offer an advantage, they suffer from the several problems. One particular way the industry handled it was through some specific technologies.

Integration technologies

image

There are several different technologies that evolved to support integration. At their heart, they support integration at data level (EDI, MDM, XML…), at application level (SOA, web services), at process level (BPM), at front end (portals).

In addition to technology standardization, these tools and technologies bring governance, standard ways of bringing in differentiation, and patterns of usage. In effect, the systems of differentiation making differentiation a well-understood, engineering led process.

So, let us review what the business operating platform is:

image

  1. ERP’s are still the center of the IT offerings, both in terms of budget and strategic importance.
  2. Differentiation is achieved through special purpose technology platforms and tools.

Pressures of disruptive models

One of the big trends in 2010’s is the way conventional wisdom has been upended: until then, technology was supporting business. It was essentially codifying what business wants in IT systems. It was helping businesses scale. For instance, if you have an insurance company in 1950’s (I am reminded of umpteen movies set in 1950’s, starting with Apartment), however well you are doing, if you try growing, you end up with unmanageable mess.

The scaling of business systems was not particularly disruptive. It is a labor saving device, which helped the growth of organizations. The fundamental capabilities of business didn’t differ – only the ability to offer to lot more people was the new addition thanks to Technology.

Last few years we have been seeing some fundamental shifts. I suppose that Internet started it all. For the first since computers appeared in the enterprise, they started impacting the business fundamentally. The reason is simple: the consumers of the services and products, the employees that provide these services and products, the management that plans and executes the activities, the financiers that oversee the use of the assets, the investors that learn and invest in the companies – all of them are using computers in innovative ways: to learn, to socially interact, to carry out tasks, to communicate, to organize – and do all of these in highly connected devices.

For instance, when we see an Unbank like www.simple.com come up, or an Unhotel like www.airbnb.com come up, we have to question ourselves: How do these disruptive models impact the IT of the modern organizations?

Transforming the business operating platforms

Imagine if we are building a modern insurance company. How do you build it differently? First of all, the standard services IT provides are not much value. Think payroll – do they even need it? Can they get a company take care of it for them? Email system? They can use gmail. Marketing? They can use mailchimp.

In fact, for most common needs, the enterprise really doesn’t have to spend money on IT. Take a look at the kind of tools that are available to a startup today: http://steveblank.com/tools-and-blogs-for-entrepreneurs/ and http://startuptools.pbworks.com/w/page/17974963/FrontPage . Any IT department would love to offer those facilities.

Here is the way I think the role of IT is changing:

image

What we observe is this:

  1. The role of IT is moving away from standard operations.
  2. ERP is only partially outsourced. For the reasons of security, control, and integration, lot of intertwined functionality is retained under IT.
  3. The Business Operating Platform that IT develops and controls is steadily shifting higher.

How does this new IT address the disruptive business models? Not yet. These systems only address the differentiators –the ones that help them compete with other businesses that are similar to them.

If you look at the disruptive models, most of them that are coming out of startups or innovative spaces are around a few ideas:

  1. Reach the consumers in new and innovative ways: For instance how instagram or hipmunk do it.
  2. Get more information out of available data:  LinkedIn, Groupon, NetFlix leverage the data and precisely target the customers.
  3. Understand the market better: By working well with the social universe and creating the right market for the services. For instance, several FB based companies do that (example: Zynga).
  4. Go after the long tail: Organizations have been focusing on serving the statistical median that they are losing the growing long tail (hipsters, anyone?). Several companies, thanks to the reduced costs of services and IT, are able to create a market for the long tail.
  5. Simplify the offerings: Some companies (actually, Google did it), are able to take a moribund sector and simplify the offerings to revitalize the market. For example, staid telephony market is shaken by DIY company www.twilio.com.

This list is not comprehensive. In fact, it is continuously growing. The hallmark of these ideas is not how they are implemented, but how they are disrupting the way the world works.

image

As you can from the above picture, the standard operating platform business is incorporating the systems of innovation as well. It is not yet clear how this will eventually evolve, but for now, there is a great deal of excitement.

Another way to look at the picture is how the BOP transformation is evolving:

image

The evolution of the BOP towards the innovative space and the increasing emphasis IT is placing on it is indeed interesting. For a full description of the architecture perspective of the same, refer to: http://www.kanneganti.com/technical/it-transformation-an-architecture-perspective/.

Of course, there are lot of open questions: How do we build for something that is constantly changing? If we want to run the business, how do we make use of systems that may be in flux? If I make my web applications mobile, am I transforming the IT? Someday, I will get to all those questions in this blog.

Conclusion

Let’s recap once: we started with the rise of modern ERP systems. We traced the need to create flexible applications and the rise of integration. We later showed how the differentiation begat new technologies that came to address extension or customization or filling-in-the-gaps of an ERP system.

image

The modern business operating platform looks more interesting than before. As IT sheds unnecessary, non-value-add tasks like productivity applications, it is freed to create more innovative applications. As shown in the picture above, IT needs to learn not only new technologies, but also need to understand what needs to be done. Whether IT can step up to it or not, I cannot say.

Ob Plug: My group in HCL (Enterprise Transformation Services) has been working on the modern business operating platforms for a while. We have tools, frameworks, sample platforms etc to support the POV expressed here. In fact, BOPT™ (Business operating platform transformation) is one of the main service offerings from my group in HCL. If you would like to know more, shoot an email to me and I will put you in touch with right people.

 Posted by at 6:58 am
May 072012
 

“Until you can measure something and express it in numbers, you have only the beginning of understanding.” – Lord Kelvin.

There are lot of things we want to measure, even if they can’t be objectively measured. For example, we measure customers satisfaction – there are no scientific units to use for the measurement. We measure popularity with proxy metrics like number of Google hits. Measuring anything objectively is itself a challenge.

My first company, Savera Systems Incorporated, had stellar advisers.  One of them, Jeff Ullman told us the following: “Whatever you measure, you improve that metric”. I observed that to be correct. It is our tendency to improve whatever we measure. But, it is not the complete story.

image

Let us say your CEO laid out the vision for the company:  “becoming a leader in the online sales of  hair tonic”. The goal could be “increase the sales by 100% in North America”. Now that we measure the sales, do they automatically increase? Unfortunately no.

There were a series of tests conducted by education psychologists in 2005.  In some cases, they paid students money for getting good grades. In some cases, instead of grades, they measured the number of days absent, number of assignments turned in, number of books read etc. What they realized was that there was no effect on the grades when they paid for the grades. But, there was positive correlation when they paid for attending classes and reading books.

image

When we measure people on what they control , that has a positive impact. When we measure on what they can’t control, there was no impact. Attending classes is something that the students controlled; getting grades was not something they controlled. So, we convert what we want to measure to proxy metrics.

So, let us say that we measure what they can control using these proxy metrics. Surely that helps our cause, right?

In the last few years, we realized that measuring the company performance without regarding the constraints (legal and ethical) had lead to long term damage to the companies and the world. Suppose we pay students for grades, what prevents them from cheating? Unless we impose the constraints, the metrics will become meaningless.

image

Let us say that we impose the proper constraints, will it be sufficient? Again, no. The problem is that sometimes by the time we got the measurements, it is already late. For example, let us say that we are measuring the sales performance to increase the revenue. By the time we measure the performance, we lose the ability to control the outcome. There is no chance for course correction.

image

So, what we do is to measure the leading indicators. These indicators portend what is going to come. For example, building permits for new construction is a leading indicator for the economy. In contrast, average prime rate is a lagging indicator. The indicators that move with the main metric are called coincident indicators, which we don’t have to bother about now.

The issue with leading indicators is that they are not always good indicators. They are often not well-defined; their correlation with the main indicator is not well-understood; there is a chance that the correlation could vary. Still, with all uncertainty, leading indicators offer better chance of reaching our goals than coincidental indicators.

We are not done yet. Suppose we are measuring the school performance by attendance. The principal incentivizes the students for attending school. Then, even the people who are not interested in school attend it. In fact, it may be possible that they disrupt the school so much that the school performance may go down.

Or, consider the case of paying for the number of bugs fixed. This payment may lead to perverse incentive for introducing trivial bugs or even breaking down a large bug as several sub-bugs. In fact, it is fairly common in medical industry to break down a single problem as a series of several ailments, each of which is separately treated and billed. What we are seeing is “unintended consequence” of a good intention.

image

That leads us to selection of several metrics that covers the desired outcome. These metrics collectively support the desired outcome without leading to unintended consequences. Since there can be many such metrics, we can stipulate the following rules:

  1. The metrics should be independent: Think of as vectors which are orthogonal. Otherwise, these metrics end up duplicating the effect. This is easy enough to validate, either empirically or even sometimes through modeling the problem.
  2. The metrics should be few in number: If we measure too many, the complexity of measuring overwhelms the people to understand what is being measured. This is often the cause for people not showing enthusiasm for metrics.
  3. The metrics should cover the original desired outcome: How can we be sure that we got all the metrics covered? That is a difficult problem to solve. It is more of a craft than science. I suppose we can choose large enough number of metrics, but that messes up the earlier rule of having few metrics.
Note for enterprise architects

When enterprise architects plan the IT activities for an organization, they create a strategy, a plan to realize the strategy, and a program to execute the plan. Unless they create the metrics at every stage they will not have traceability for the entire program.

image

There are several artifacts that we use in this process:

  1. A model to translate the metrics from one stage to another.
  2. A correlation mechanism to establish the relationship from one stage metrics to another.

Without going into full details, here are some tips for this methodology:

  1. Goals to KRI (Key Result Indicator)’s: While the organization goals can be nebulous, KRI’s have to be precise. They need to establish what we are measuring, how we are measuring, and who the people are that are responsible for those KRI’s are.
  2. KRI’s to program metrics: While KRI’s are precise to measure, they tend to be lagging or coincidental indicators. What we need are leading indicators. At a program level, we can establish leading indicators, simply by identifying the indicators that we can measure while the program is in progress. For instance, when we construct a program, we typically break down the program into multiple, simultaneous projects. By establishing the metrics that can be measured throughout the life cycle, we are creating the simplest leading indicators.
  3. Establishing code level metrics: Eventually, we should focus on automated creation of metrics. By incorporating into code, we ease the process. For instance, if we are trying to improve customer satisfaction, we might measure the number of interactions that a customer had to have to resolve the issue. Or, the duration of the open issue. Or, number of exceptions that service reps had to take. Of course, the earlier part of the essay described the process of choosing the metrics: for completeness, independence, and with constraints.
  4. Establishing the feedback loop: As long as a we have a model that correlates the events at each stage, we can keep refining based on the information from the ground. The idea is that we end up with manageable number of metrics that can predict the final outcome reliably.

I have not found a good book on this subject that provides a good mixture of management processes, Enterprise architecture processes, mathematical models, and behavioral psychology. I wrote this piece entirely from my experience. If anybody knows some good resources on this topic, please inform me via the comments.

 Posted by at 9:59 am
May 022012
 

Long time ago, I was blissfully unaware of IT technology. I designed programming languages, developed compilers, architected new database technologies, and dabbled in operating systems. Life revolved around all the things I learnt in school and in the labs: computers, algorithms, applied science.

Somehow, after I ended on this side of the fence, all that changed. Usually, it is implementing a packaged application, rolling out an ERP, supporting large number of IT applications. It is challenging – it brings in knowledge of finance, business, planning, organizational behaviors and whole lot of other things to make IT work well. Even though I never signed up for doing an “IT” job, I drifted into it and found myself enjoying connecting what I learnt and what I see.

Recently, I have been seeing a great deal of changes in IT. It is very similar to the changes in IT from late 90’s when the internet hit the scene. Yet, it is much more – this time, there is a fundamental shift in the way IT is being done. Perhaps, these words are too easily thrown around. Let me make my case.

Traditional IT technologies

First let us take a look at the traditional IT technology:

image

This is what IT has been doing in the enterprises. Once some need in the business is well-established, IT builds an application (preferably customizes a packaged application or builds on top of some vendor technology product) and operates in their data center using their standard rollout and support processes.

Influences on IT technologies

Think about the history of modern organizations. Just when organizations couldn’t grow profitably, computers made the growth possible with information technology. The constraining factor in growth was information management (gathering, organizing, retrieving, and analyzing). Once tamed, it allowed the organizations to grow and take advantages of their size.

IT, of course, helped that growth. However, now, it is holding back the enterprise. CIO’s have very short tenures. They preside over moribund enterprises. The users see that their IT is offering poor services at higher price, with the illusion of security and control. [Think gmail and your organization mail. Who offers more storage? Who offers better searches, better backup, and better access?]

There is a specific way of understanding the impact on IT technology from three directions (the idea for this classification came from Betsy Burton – a shout out for her classification):

image

Each of these technologies (from business, consumer and operations) impact the IT technologies in a specific way (and they are influencing one another, and in particular, the business technology as well).

Business Technology

When wall-street journal starts writing about technology for business people, you know that there is something up. These days, you can’t turn a page without seeing something about big data and how business is trying to make sense out of this data. Traditionally, IT did not deal with this problem. This esoteric field of decision support systems is managed by a close group of business and experts.

Normally, these class of applications undermine an IT department. If a marketing program needs to be run, if sales data needs to be analyzed, if a partner needs to be engaged, business reaches out to its own resources, instead of IT department. There are specific reasons why:

image

There are several kinds of applications that business uses that IT has reclaimed over time. For some applications, IT does not have a good hold yet. Here are a sample applications:

image

A trend from the last few years has been “Business aligned IT”, where these kind of business applications have been a large focus. Already BPM, CRM, E-commerce applications have become an integral part of the business systems. Some of the new kind of applications such as big data, advanced analytics are only being talked about. The marketing applications are still too much in a flux to become a part of IT yet.

Consumer Technology

There are several times consumer technology made a big impact on the IT. Remember PC? Internet? All of those started as consumer technologies. Of course, It was reluctant to embrace those technologies at first. But as the IT users were using those tools at home, IT itself had to offer them as a part of their portfolio.

The current consumer technologies that are impacting the IT are many:

image

These are not just the application categories, but aspects of applications. Take instagram, for instance: it has mobile computing; it uses social networking; stores the images in the cloud; it is a customer centric application. Or, take Quora: it is a social networking application; it is for knowledge management; it is collaborative.

As mentioned earlier, while the main purpose for IT was information management, over the time, the bottleneck ended up being the way people work together as a team. While the computers are good at information processing, they are so good at capturing the information from people and letting them work as a team. Scaling of the human productivity was aided by different techniques overtime: using energy in industrial revolution (steam engine); using assembly line in manufacturing. The current techniques let the corporations harness the power of large crowds, which is exactly what consumer IT offers.

There is another impact (consumerisation of IT) – where the consumer devices end up being used in IT (personal computers, cell phones, and smart devices). That has profound impact on one of the core functions of IT (desktop support etc).

Operational Technology

I like to tell a tale how electricity distribution happened. At one time, can you believe that factories ran their own power sources? Once a reliable grid came up, it was far cheaper to produce and distribute energy. Perhaps our future generations will tell the tale same way: can you believe that companies ran their own datacenters?

Cloud and related technologies are impacting the IT in transformational way. There are several themes to this transformation:

image

One of the more exciting things to come out of operational technologies is “devops” which integrates the continuous development with operational view of the applications. Even if the companies are not moving to cloud, these best practices transform the IT.

Summary

IT is trying to address the needs of business by offering more and more applications built on business technology. To get better adaptability and to increase cooperation, they are adapting consumer technology. To reduce the costs, and focus on the core competency, they are looking to operational technology such as devops and cloud.

 Posted by at 8:40 pm
Apr 292012
 

People say that history is written by the victors. Actually, history is written by the scribes. I suppose they can portray themselves as the victors – which makes the sentence true, in a way.

What I am leading to is this: if you are the note taker in any meeting, you control the outcome. Nobody remembers what happened in the meeting after two months. If you are the person who took the notes, you have the ability to change the perception of the meeting – without falsifying the information. You can create the right perception of the outcomes, action items and so on.

My pet peeve in most meeting is that people do not take notes. They rely on faulty memories to recreate the moments later on. They miss out the essential points. They miss out the nuances. They only remember what they think happened. It is like an Agatha Christie novel – nobody will remember all the conversations.

Here is my prescription for those problems, especially in the meetings where the agenda is fluid. There are two problems in meetings, especially, exploratory meetings:

  • Mismatch in the understanding: The meeting may not offer us a chance to understand and restate the problems as we see them.
  • Lack of follow up: While we understand what that is said then and there, as soon as we walk out of the meeting, it is jumbled up. Even action items may not help, as the context is lost.

I find it useful to use note taking as a way to take care of these problems. Here are a few simple note taking best practices:

Always take notes on a computer

In this day and age, you don’t want to use a notebook and then transcribe. If you are anything like me, you will never end up writing and enhancing. In fact, the more you practice taking notes on the computer, the better you will become. Imagine the effect you can create by sending out the notes and your thoughts immediately after the meeting!

You can use any tool for taking notes, but I prefer freemind (or XMind) as it is free. It is a mind-mapping tool, that provides a hierarchical view of the information. It is especially useful, when the information is hierarchical instead of being linear. For instance, if you are listing out all the team members and their role and other details– this information is not linear (you don’t care which order you get to them), but hierarchical (you list out the name and under the name you list the role and other details).

Start with a simple template

As I said, I prefer the mind map tool for this job. Before the meeting, I jot down the various aspect of the meeting that I am supposed to gather information about. (If I am providing the information, that is a different kind of meeting).

Take this example: Suppose we are having a meeting about a project we want to start.  You are meeting the customer as a potential project manager.  A possible information template could be:

mindmap-sample

Of course, your headings could change. The beauty of such arrangement is that you can see the items of similar importance at the same font size and same distance from the center (Radial Hierarchy).

Keep the hierarchy in mind – use it to guide your questions

As the meeting progresses, you find yourself doing the following:

  1. As you get information about any topic, you will put it under that topic. Good – this is the way to go.
  2. You are getting details about a topic, but the context is not there:  What it means is that you are at level 1 and you are getting details about level 3. You will establish the context at level 2. Example: You may be given details about the project users, without categorizing the details. That means you will supply the categorization (like business users, consumers etc.).
  3. You are getting new top level topics: That may mean two things: you may not have thought about the meeting to give a good starting point. Or, the meeting is going off direction. In the first case, you add the topic to your mind map. In the second case, you nudge the meeting to the right topic: (“Before getting to those details, can you please tell me who are the people involved so far in the project?”).

If all things go well, you will get a rich description of the meeting. And, you will look like a genius for providing a structure and context to the information.

One advantage with mind-map tool is this: By looking at the picture, I can tell if we did a good job on information gathering or not. If we are too deep in one topic, the image shows the imbalance. If we did not cover a topic or excessively covered a level 2 topic, the picture clearly shows.

Publish it immediately

You think you will refine the mind-map, create a document and then publish it. Trust me, you won’t get around to it. I suggest you correct the typos, enter any contact information, clear out any questionable material, pay attention to the action items, and if needed, add your perspectives (make sure that is marked as your take on the meeting), and then publish it immediately. I publish my mind-map meeting minutes in less than 30 minutes after the meeting, in general.

Summary: Always take notes if you are participating in an exploratory meeting. Use a computer and a program to take the notes. Prepare an outline and use it to guide the meeting. Publish the finished outline.

Next steps

Here are several ways you can innovate on the basic theme:

  • You can create standard templates for different kinds of meetings. For instance, pre-sales, sales, new customer, existing customer, project status – meetings for all these different contexts can be driven effectively with customized templates for mindmap.
  • For different kinds of meetings (say you are attending a conference) you would use different tools. For instance, if it merely reporting need, I resort to Word or simple text editor.

Always try to see how you can capture in the computer itself, as it reduces the barriers to making the notes available digitally.

 Posted by at 9:22 am
Apr 232012
 

I am a developer with questionable credentials in aesthetics with one saving grace – I recognize when I see a good one in design. In developing web sites, I have explored a few options for web design and benefited from some of them. A few of my readers have asked me questions about these options and I decided to write a note as a response. I am also structuring the note as a lesson plan so that any newbie can benefit from it. (If I had time, I could have created a course at Udemy).

Objectives

I am assuming that these are the objectives that you have.

  • You want to create good looking websites.
  • You do not have access to designers from beginning. You may be able to rope in UX designers later in the game, but to start with you still want a good looking site.
  • You prefer tried and well understood interfaces instead of creating a unique look and feel.
  • You don’t mind learning a bit of new technologies to go with code-Fu.
Prerequisites
  • You should be able to develop websites – you should know basics of HTTP, HTML and so on.
  • You should know a programming language to program the web (I assume you are a J2EE developer or PHP/python/ruby developer).
Motivation

Let us say that you are a developer out of college. You went through your courses and graduated with a good bit of CS knowledge. You are working for a large company. You are developing a web application that is heavy on the backend with lot of potential to save the company some money through process optimization. This application is for internal employees.

After the first prototype, you are asked to show the prototype to the big boss. You sweat it out and create an end-to-end scenario and show it him. Unfortunately, the first thing that catches his attention is the color scheme or the misplaced logos. Then, the complex flows and intricate forms. From your point of view, you keep thinking “why is he focused on these? Doesn’t he know that we will get a UI designer and fix it before the release?”

At the same time, a college intern working for the summer creates a wordpress site. Naturally, it didn’t have the end-to-end scenario that you have in mind, but it has lot of other niceties that the big boss cares about: colors, nice flow, ability to create a social group, interactions and so on. The big boss asks you why you can’t put together a professional demo as good as the one by the intern? Can’t a team beat a single intern?

You look around and you realize that the web has amazing number of websites that are well-designed (at any rate, better than yours). Seemingly, they are done on shoe-string budgets, without using costly skills. Your manager looks at this market and sees that typically this process uses the following kind of people:

  • UX designers: These people design the pages in traditional media like PSD (Photoshop files).
  • HTML designers: They convert PSD to well-designed HTML

From there onwards, the developers can use the HTML templates and code to their specs. Turns out that both these two skills are commodity skills (You can get a a 10 page – consider each page to be a template – site done under $2000).

But then, the manager recognizes these issues:

  • He needs these skills through out the project. Outsourcing doesn’t work as well.
  • There is a considerable amount of interaction. Unless somebody in the core team doesn’t understand these skills, they end up paying for the mismatched execution.

So, realizing that he needs some of these skills in-house, the manager asked you to look into it. Your next task is to understand enough to create a good looking site and involve the designers when you need them, and get the most out of them. You also decided to use couple of junior developers in this process as well.

Where do you start?

Skills you need to learn

To develop good websites you need to know the basics of:

  • HTML5: This skill helps you to design clean HTML sites.
  • CSS: This skill lets you design good looking websites.
  • JavaScript (JS): This skill lets you program the web to be interactive.

Of course, knowing these will not help you to make good looking websites. You want to understand the reusable patterns, standard usages, and the conventions. Thankfully, several frameworks help you to learn those:

  • JQuery: It lets you control the webpage look, content from JS easily.
  • Sass + Compass + ZURB Foundation: The first two technologies help you to tame CSS – not that they are important, but if you want to use some advanced frameworks you might need them. ZURB helps you to create a good looking web page templates from some simple specs from you. Very popular with Rails crowd.
  • Less + JQuery + Twitter Bootstrap: Less is an alternative to Sass which lets you generate CSS from simple macros. Twitter Bootstrap is amazing template for developing websites.

At the end of all this, you will easily put together pages like:

image

and

image

without having to worry about a lot of details.

Learning the pre-requisites

Currently, the best source to learn (by practice) is code academy. If you don’t skip your lessons and practice all the projects, you will not only learn, but also retain the knowledge about JS, CSS, and HTML. Once you go past, here are some specific resources for each of the topics:

HTML5
  •  Paul Irish’s amazing HTML5 Boiler plate. Not that you want to start your web pages from there, but you will understand the anatomy of a HTML5 page.
  • http://www.htmlfivewow.com – be sure to use it from a modern browser like Chrome or FireFox 11 or IE 10.

Pointers about learning HTML5: Don’t spend too much time on it. You will learn as you practice, once you get the hang of basics.

Exercises

For practice, do the following:

  • Create a slide deck using the template provided in HTML5wow.
  • Take your company webpage and redo it in HTML5. You don’t have to adjust CSS or anything yet. Just, restructure it to use HTML5 boilerplate.
JavaScript

If you practiced using code academy, that is teaches you enough about the language basics: primitives, loops, functions, prototypes, objects and so on. Here are some additional resources:

Exercises

For practice, do the following:

  • Solve 8-queens problem in JS. Print the results to HTML screen. For additional marks, learn how to display it properly on a chess board.
  • Solve the same problem, using underscore.js as the utility library.

Learning the frameworks

JQuery:

The power of JS is that it can be used to manipulate DOM (a structured representation of HTML) in the browser. JQuery makes this manipulation simple and succinct. Here are the resources to learn JQuery:

Still, the best way to learn JQuery is to develop a site with it, which is what you will do next.

Exercises:
  • Take your corporate website (or any generic site, that is not interactive). Redo it in JQuery in the following way:
    • Assume that the users are logging in (so, you have their username).
    • Get their twitter feed via Ajax and display it in a different tab.
    • Implement the navigation, screen design and such in JQuery (and JQuery UI).
  • Do the 8-queens problem. This time, as the search is happening, animate the board slowly to show the inner workings trial and backtrack model.

Twitter Bootstrap

You can learn twitter bootstrap by playing with it. Still, the following are good resources:

If you went through the earlier lessons, you do not need to know much about bootstrap. In fact, you don’t even need to learn JQuery to start with bootstrap. So, if you want to start with bootstrap, you can do it along with JS learning.

Exercises:
  • Take your corporate website. Redo it with bootstrap.
  • Take plus.google.com – do it with bootstrap and see how it looks.
  • http://ajkochanowicz.github.com/Kickstrap/ – try this with the corporate site. In particular, use the icon fonts.

Note: Any webpage you developed using this framework renders well on mobile devices. It follows the principles of “responsive UI”.

Final test

If you made it so far, you know:

  • basics of HTML5 and CSS
  • Good knowledge of JS and JQuery.
  • Good website starter toolkit (including less, coffeescript(!) and so on).

To test yourself, consider developing one full website using the following technologies:

  • Use bootstrap for the page layout and the page design.
  • Use JQuery for the DOM manipulation
  • Use underscore.js for JS utility library
  • Use Require.js to manage the JS inclusion.
  • If you are serious about it, you can try to use backbone.js as the MVC framework on the browser.
  • You probably will need some sever side services to implement any decent website.

For problems, you can pick any of these:

  • A reporting system where people enter reports on their activities. Different people get different views. Different reports are generated for mailing periodically.
  • A chess game web site where two people play against each other. Make any assumptions that you need to, to make the system simple.
 Posted by at 4:34 pm
Apr 102012
 

Ok, I can’t tell you how to do that, but here is how Instagram did it. I am not going to talk about how they understood their customers and how they created something customers loved. That is all marketese – I can’t tell you how to replicate it.

What I can tell you is that they only have three engineers supporting all their webops to take care of billions of photos, terabytes of data, and millions of users. The numbers are mind-boggling. If these numbers are thrown at any CIO of an enterprise, they would come back with a budget for 100 people and 3 year plan to implement a program to manage the data.

Here are a few simple things they did right:

  • They only focused on essentials – they did not focus on keeping anything in-house that did not belong there. Yes, they are entirely cloud based. They are heavy users of Amazon EC2.
  • They used open source extensively and hacked it when needed.
    • They use Ubuntu 11.04 on EC2
    • Django for app server (stateless web – means horizontal scaling).
    • Stripped down web server (normally it is apache + mod_wsgi for python, but for their needs they needed low CPU webserver and therefore, they used ‘Green Unicorn’ (a Python WSGI HTTP Server)).
    • PostgreSQL for database (sharded cluster with 12 replicas in different zones)
    • Amazon S3 for photo storage
    • Amazon CloudFront for CDN
    • Redis as in-memory storage for feeds
    • Memcache for caching web service support (not sure why did not use Redis here also – most likely the software already works with memcache).
    • Apache Solr for searching (with JSON interface)
    • Twisted for pushing billions of notifications
  • Good focus on DevOps
    • They used nginx for load balancing (see my proposal for earlier).
    • They used Amazon Elastic Load balancer (though, they could do without it).
    • Munin for monitoring
    • Outsourced services for incident notifications(Pingdom for monitoring and PagerDuty for incidents)
    • Sentry for App server reporting in real-time

     

Slide21

The picture is a rough approximation (most of the information is taken from the wonderful site: http://instagram-engineering.tumblr.com)

What lessons lie for us poor enterprise developers, who are stuck using Java, and forced to use in-house resources that are neither flexible not scalable? Unfortunately, we will have to wait until the IT people let go of their cold dead-fingers off the inflexible IT.

Nevertheless, here is what an architect could do:

  1. Architect the systems such a way that parts of the resources (data, especially) lies outside the enterprise.
  2. Use open standards like REST and JSON to quickly pull together different systems
  3. Focus on DevOps from the beginning. Assume that your application needs to be maintained.
  4. Keep a consistent set of tools (most of the tools used in Instagram are popular in Python community)
  5. Most importantly, focus on getting the job done!
 Posted by at 11:20 am
Mar 092012
 

Last weekend, I was optimizing a site, to demonstrate what we can do, for a customer. I thought I would write about some of the basic tools that I used, since most of the junior developers do not know understand the full potential of these tools.

The two tools I reach out to test the web applications are Firefox and chrome. Chrome has good basic tools for performance testing, but Firefox shines with its many plugins.

First start by downloading the basic Firefox and installing it. Then, install the following plugins:

Firebug and friends

By far the most useful plugins are based on Firebug. Here are the ones I use:

  1. Firebug: Coolest plugin. You can see any element of the web page and understand what it is, what is the style applied, how the layout works etc. You can even dynamically edit the page to see how it looks before making any HTML changes back on the server.
  2. Friends of firebug: These ones are mostly cosmetic plugins that depend on firebug.
    1. friendly bug
    2. firebug autocompleter
    3. Firecookie
    4. Firefinder
    5. Fireflow
    6. FireRainbow
  3. Yslow: Understand why a web page is slow
  4. Pagespeed: Google’s version of why slow plugin

Here are the typical use cases:

Say, you want to see what styling a particular element has (for example, you want to see what font is being used). Here is what you can do:

image

Select the element and choose “Inspect element”. You get to see the HTML in one pane and style in another. Notice how the style part shows which css has impacted the style and how. If you want to see what is being overridden, this is the best tool.

Or, you want to examine how the HTML corresponds to the elements of the page, fire up Firebug and you will see the details in a separate pane. You can select an element and see the style, computed style, layout, and the DOM.

image

Here is another little trick. You can edit any of the HTML or CSS you see and instantly figure out the impact on the page.

Suppose you want to know why a page is slow. All you need is to run the Yslow plugin (which makes use of Firebug). Here is what you see:

image

You will see several suggestions to improve the performance of the site. Most of these suggestions are valid – but, it takes some expertise to interpret the results.

A similar view is provided by Google’s pagespeed:

image

I am not showing all the details, needless to say. Except for a couple of differences, it is similar to Yslow.

Developer Toolbar

While Firebug is interactive and nice, there is an older plugin that fulfills different needs (Web Developer). Here is what it can do.

  1. You can disable the elements of the page selectively – Javascript, CSS, etc. That tells you more about how the page looks.
  2. You can manage cookies (there are other plugins that specialize in cookie management – but this one suffices for most needs).
  3. You can see CSS about an element, edit it, and disable it etc. It is similar to firebug, but a little simpler to use.
  4. You can manage the forms – you can see each element description, layout, make the hidden fields visible, make the password fields readable etc.
  5. You can check out the images – sizes, positions, alt tags etc. Useful to see those web bugs.
  6. You can get various pieces of information about the page: layout, block details etc. It actually overlays the information on the page making it easy to understand for the novices.
  7. You can validate the page against several rules: Sec 508 compliance etc. You can validate HTML, CSS, links, images etc.

It is a neat little toolbar that is easy to use (firebug is more developer friendly; this one is user friendly).

HTTP protocol analyzers

Suppose you want to know all the requests going back from a page (especially these days with all the AJAX calls, you can see all those requests using these tools. I use the following tools:

  1. Live HTTP headers
  2. HTTPFox

I like HTTPFox for the better control. Here is how a facebook request might look like (View –> HTTPFox is the way to invoke it):

image

You can see the summary, and the details of each request. This is wonderful way of seeing how the CDN is working, caching is working, and the cookies are working.

Misc

Here are miscellaneous ones that I use:

  1. FireShot: You don’t need the Pro version. The free version is good enough for us. And, it lets us annotate the screenshot too!
  2. LightShot: This is a simple tool – not a commercial one.
Things Indian

These two plugins are no way needed for development. However, I use them often to help read and write Indian languages. Considering that they use something I invented (Telugu Transliteration scheme called RTS) 20 years back, I am particularly gratified. Here they are:

  1. Indic Input Extension – it lets you type Indian languages in English in normal keyboard.
  2. Padma – it lets you transform a page written in in romanized Telugu to Telugu script.
Summary

With these plugins, we can analyze a page, optimize a page, and transform a page. I have setup a collection called: https://addons.mozilla.org/en-US/firefox/collections/kramarao/deve/ so that you can download all of them in one go.

 Posted by at 10:28 am