Thursday, March 18, 2010

The agile experience

There are hundreds and thousands of literature available on internet about agile methodology. So, I’ll not dive myself to explain what it is and what it is not. Rather, this blog entry is purely my own experience on agile methodology. Couple of years ago I joined Financial Market System Development team of Standard Chartered Bank in Singapore and I was introduced with agile process. After working with traditional waterfall model for last few years, agile was like a fresh air to breath. I was so excited that I named my blog as “agilej” and in my dream I was about open a fast food chain in India with a name of agile food (read fast food!!).


Jokes apart, it was quite interesting to work with a model which has some pragmatic ideas to overcome the drawbacks of waterfall model. But, as we know, nothing is perfect in this world. If there are something wrong in waterfall, then of course it has good credentials as well, otherwise the majority of IT industry would not have opted for it. At the same time, agile is not Mr. Perfect. It is very young approach and there are so many speculations about it. The choice of the approach completely depends on what kind project we are doing, how much funding we have and how many good and smart developers are with you. Those factors drive the decisions to go with waterfall way or agile way.




Before I start, here are list of abbreviations I’m using in the rest of the article:-
  1. IT      :- Information technology
  2. BA     :- Business Analyst
  3. RA     :- Requirement Analysis                           
  4. BRD   :- Business Requirement Document
  5. FS     :- Functional Specification. It is enhance version of BRD, typically written by BA.
  6. DD    :- Design Document          
  7. HLD  :- High Level Design
  8. LLD   :- Low Level Design
  9. CR    :- Change Request
  10. UML  :- Unified Modeling Language
  11. CI     :- Continuous Integration
  12. TDD :-Test Driven Development
Age old waterfall model is mature enough and there are some visible pitfalls. I’ll address on by one.


  • The complex team structure is the main bottleneck. It opens lot of loopholes which can be political instruments within the project teams. Normally, in technical side, we’ve team lead, module lead, technical analyst and many more jargons you can imagine. The developer is the poorest guy in the team, who is treated like code monkey and although they are doing one of the important job, they hardly get the due respect, either in terms of pay or appreciation. 
  • The mode of communication is liner in waterfall model. BA is the middle man between Developers and Business. Developers hardly get the opportunity to interact with business people. That is a huge gap in the process. The job of BA is very critical, who translate the business requirement into something which the user want to do by click of the mouse. Unfortunately, the translation is done correctly in many cases.  I’m not saying that BA is incompetent, but the communication will be much more productive if you include developers in all the decision making process. Business people may need some functionality which may be technically challenging; the developers are better person to understand than the BA.
  • What is the difference between project manager and project lead?  If you google it, you will get some theoretical answers. But practically are those roles are not identical? That is also one of the problems: too many managers. And here agile is very lean.
  • There is no doubt that documentation is as important as the binaries. But, the waterfall model put so much stress on documentation, that we are drifted away from the real software product. Documents like BRD, FS, DD (some even go for HLD, LLD) can be done in smart and lean way. We do not need so much garbage. UML is the best tool to document all the necessary artifacts. That will reduce hundreds pages of FS into few sheets of use cases, activity diagrams, which are easier to understand.
  • The initial phases of waterfall model viz. RA and design are treated like honeymoon period of the project. People just talk and talk and write something on the functional specification, which changes after one week. Fortunately some organizations do some static prototyping of the delivery, but most company depends on the abstractions of BRD or FS. It is assumed that other parties involved in the project will understand that collection of sheets. And honestly speaking that happens very rarely.
Now, enough shit about waterfall model. Time to give some claps for the agile process:-
  • In a typical waterfall model, the business gets the taste of the software in UAT, where user will test whether the product is acceptable or not. I worked with a bank before with a huge retail banking project. It took almost a year for RA, six months of design and six months of development. So, a requirement which was quoted by user one year ago in the BRD, he can see it after twelve months. And hardly there are cases that they will be happy. In today’s financial word, one year is a huge time. So, if the user is not happy with UAT (which eventually happens!!), a lot of extra pain is gifted to the project team. Agile is more pragmatic in this regard. In agile practice, there no clear separation RA, design and development. Unlike waterfall, agile deliver a workable running piece of software in an iteration. Iteration could be of one or two weeks, depending on the complexity of the delivery. So, after user came up with a requirement, the BA and developer can deliver that in iteration. Business can test it and give their feedback. BA and Developers work together to implement that. So, this cycle will reduce the chances of getting surprised.
  • CI is very powerful feature of agile. Unlike waterfall, where a separate team is created to look after build and deployment process, agile automate it by using CI products like atlassian bamboo and many other. As I told, agile deliver in iteration and believes in software as evolving process, we need to deliver very quickly. We can configure the frequency of build and deployment
  • Agile gives importance to people over process .Unlike waterfall, developers get the deserved respect. They are part of the important decision making process and can talk with the business people directly, if needed.
  • Nice collaborating platforms like Mingle or JIRA are very powerful to show the current status of the project and who is supposed to do what. Every stakeholder of the project and login to system and can see what is on his plate.
  • Agile community vividly speaks to replace heavyweight BRD/FS by UML diagrams and unit tests. Tools like Enterprise architect are very powerful which can be used by any stake holder of the project. Also, TDD approach is very practical where the unit tests encapsulate the little tiny functions. There are powerful tools available for code coverage, static code analysis to produce nice and healthy code.


Some people in IT community see agile as a revolution in software development process, something like October revolution. Lots of people are coming up with some new ideas to shape agile process. Some says, we don’t need any testers, unit tests are enough; some says don’t do documentation at all. In http://www.infoq.com  you can see a lot of blogs, articles.
The bottom line is waterfall will coexist with agile. May be new development projects will give a try to the agile methodology, but maintenance and enhancement projects will keep on using waterfall model. In my opinion use agile process if and only if all the stakeholders of the project understand what it is and what is expected out of it. Don’t just jump into agile, because your friend in other company is doing it. More and more organizations are going to adopt agile in future, but we can not erase waterfall model immediately. Big firms like financial institutes, which are globally distributed, have offshore onsite model, may find it difficult to switch to agile. Although there are agile models for distributed systems, but I know how much effective they are.  Let see how the agile process is shaping up.

1 comment:

Kamal said...

Anil - Nice summary and well articulated thoughts.