Agile: Revolution or Evolution?

Agile represents a change for traditional development shops, but is it a revolution or an evolution? While Agile feels like a significant departure from past development practices and has the potential to transform productivity and responsiveness, it’s really one end of the development methodology spectrum. The other end, of course, is the classic methodology, waterfall. Agile is also a cross-view into the factors, both controllable and uncontrollable, that impact development success.

At heart, the development time frame is the most visible change between waterfall and Agile. Waterfall time frames can stretch out years. This makes sense when you consider that much of the waterfall methodology stems from the construction industry, where the final plan must be completed before the foundation is poured, since it’s incredibly difficult and costly to make radical changes in the middle of construction. If you’ve ever changed the layout of an addition halfway through, the sticker shock of the change order probably made a lasting impression. I started out managing standard waterfall projects with year-plus time frames, but I found that multiple, shorter time frames in the 3 month range were far better. Each development cycle or phase gave the business and developers a chance to review the remaining work and re-prioritize. After doing the basic configuration of a billing system, perhaps we learned that the stock Collections functionality was unusable, so we should spend some time addressing it. Agile takes this to a further extreme of 2 week development cycles, or sprints, with similar review and re-prioritization opportunities.

 Cross-views and skepticism

Agile looks at development at right-angles from waterfall and takes a skeptical, show-me stance on everything.

Predictability

• Waterfall is based upon significant confidence that what is known at the start of a long project is accurate, including predictions about future needs. “In two years when the project is finished, our business will look like this, and require that.”
• Agile takes the stance that the farther out you go, the less predictable the business can be. New regulations, new competitors, new technology, and new market realities can happen any time and all the time. If you ask me what I can predict (and develop for) in the next four months, I can give you a pretty good answer, with 75% or so confidence. In 8 months, on the order of 25%, in one year, maybe 10%. After that, why bother with predicting? Agile is a great example of the KISS (Keep it Simple…Simon) principle: Start with something quick and simple that you know you need, called the minimally viable product (MVP), and then figure out what’s really important to add after that.

Prioritization

• Waterfall is of course concerned with priorities, but typically groups business needs into buckets, resulting in multiple needs having the same level of priority.
• Agile operates under the idea that “If everything is a priority, nothing is a priority.” All business needs must be stacked together in order of absolute priority, called stack ranking; there is only a single #1 and single #2 priority at a time, etc. The rankings can change at any time for any reason, but the development team is always tasked with working on the top priority items.

Responsiveness

• Waterfall needs the requirements to be defined prior to the start of development and is highly resistant to changes mid-development. Have you ever had a hot new requirement come down the pike, only to be told there is a requirements freeze until the end of the project/phase?
• Agile has the same “freeze” requirement, but minimizes this impact by shortening the phase/freeze period to 2 weeks or so. Business moves fast, but a maximum of 2-3 weeks before a change can be pulled into development is rarely a limiting factor

Efficiency

• One of the major questions about Agile is “how can you be productive when everything is done in 2 week chunks?” Waterfall addresses this by assuming that developers are more productive when they can spend time focusing on one thing, and then moving to the next. This is true, in some respects. Multi-tasking and changing directions is very costly in terms of productivity. But so is developing functionality that is obsolete or unimportant once the system is finally released into production.
• Agile argues that it’s more efficient to build only what is necessary and in the order of its priority, no matter when the change in requirements or priority is recognized. If you originally planned to sell your product exclusively to large accounts but realized later that demand is much greater at the consumer level, you may want to focus on credit payment processing and go light on collections functionality. This doesn’t mean that there will be no wasted development, but it reduces the odds and impacts of dead functionality.

Responsibility

• Waterfall typically relies on business analysts to translate business needs into system requirements. This can tend to lead to a “Throw it over the wall” mentality and the feeling that coders are figuratively working in the dark.
• Agile mandates the separation of business roles from developers in that the business is only allowed to manage and prioritize the “What” and developers are exclusively in charge of the “How.” The developers self-organize and act as their own analysts, forcing meaningful communication between the two groups. This also means that the developers have much more ownership for the effectiveness of their solutions: there’s no hiding behind “Well, I coded to the spec” in Agile. If the developer doesn’t completely understand the business need, the developer must get answers from the business.

Completion

• Waterfall projects typically view computer systems as self-contained projects that end with a finished product, as well as some possible follow-on enhancement phases.
• Agile takes the view that the system is really part of a business process, something that will be ongoing for the life the company and doesn’t have a definable “end.” The pace of change within the system may drop dramatically after a point, but the nature of business priorities means that a system may suddenly need development attention at any time, for many reasons.

There are, of course, many other differences between the two methodologies, and the jargon and titles can be confusing. There are also more variants of Agile than one can shake a stick at, but the above ideas are pretty consistently held in the different flavors.

 A Caution

Agile is an amazing development in the software world (and elsewhere), but it’s not a panacea. It can be used as an excuse to skip planning for the future and developing manageable code. It’s imperative that even within the flexible and changing nature of requirements, long-term thinking and planning take place so that systems aren’t architected in a short-sighted manner that limits long-term flexibility and viability. Similarly, development standards and documentation should be held to extremely high standards. Systems developed with an attitude of “who cares if it’s good code, just get it done quickly” can become millstones around the necks of companies who find change to be next to impossible because every little change results in complex and unpredictable results. I’ve found that insisting on high quality development pays off not only in the long run, but also in the short term, since testing, interoperability and reliability are enhanced.

 Ultimately, an Agile transformation comes with great opportunity to positively impact the organization. It can also lead to disruption in terms of productivity, control, and culture when it’s not done thoughtfully and thoroughly. A well-executed Agile transformation can help management directly impact strategic decision-making while simultaneously ensure that tactical considerations are handled well by folks on the ground, making an organization more responsive to changing business needs at all levels.

If you have any questions about this article or want to talk about how to use agile to transform your business and development operations, contact me at 617-429-5529 or jmarcos@consultingclarified.com.