The Potential Perils of Commitment Driven Sprint Planning

Overview

Mike Cohn describes two main approaches to Sprint planning, namely :

• Commitment Driven - where the team decomposes stories into tasks, and estimates in hours (producing what I would call an “absolute” estimate for the sprint);
• Velocity Driven - where the team selects an appropriate number of story pointed stories off the backlog that equates to the team’s historical velocity (i.e. they use the "relative" estimate of the Story Point rather than reestimating for an "Absolute" value).

The key advantage to the Commitment Driven approach is the ability for the team to use the classic Sprint Burn Down chart to track the team’s progress towards the sprint goal. In addition, it also solves the awkward question of how much to commit to in the first sprint (where a historical average is unknown).

Which one should you go for?? Well, Mike isn’t on the fence on his like for the Commitment Driven approach in his article “Why Sprints Should Be Planned with Hours, Not Points” (https://www.scrumalliance.org/community/spotlight/mike-cohn/september-2014/why-sprints-should-be-planned-with-hours-not-point). He also provides us a nice little anecdote to illustrate his point:

To help explain why, let's consider an American football team, say the Denver Broncos. Let's suppose that halfway through the season, the Broncos statisticians calculate that the team has scored an average of 21 points a game. It is entirely appropriate for the Broncos to predict that the team will continue to average around 21 points a game for the rest of the season. It is not, however, logical for the organisation to assume that the Broncos team will score exactly 21 points in the game this Sunday night.

The Problem

I feel hourly estimates are potentially troublesome; moreover they are troublesome estimates that come at a cost.  I’ve recently had cause to reflect on my position, and why I have gravitated away from recognising the Commitment Driven approach as the de facto approach to sprint planning. My thought process is as follows:

Increased Planning Effort

Drawn out meetings and engaged developers are two things that are rarely observed together. In fact in general I find an direct correlation between length of meeting and grumpiness. Working through each story in detail, creating all the required tasks, and estimating them to get a reasonable accurate sprint burn down can take a reasonable amount of effort. For the Commitment Driven approach we need a reasonable estimate to ensure a good fit for the sprint, so a poor hourly estimate at this stage would negate any value in doing it at all.

In addition to the extra planning effort, your team also has the continuing “enjoyment” of regularly (daily) estimating effort to complete in hours. Once you get in the swing of things it may become second nature, but it is definitely an uphill struggle for new teams. 

Chintzy Estimates

I’m sold on story points and planning poker, and all the Wideband Delphi benefits that it brings. I accept that they are rough, but I’m comfortable with that. In comparison, the process of delivering hourly estimates is not really defined, and in my experience, off the cuff estimates given in a meeting are generally the least accurate ones. With the likes of #NoEstimates movement and lean, I feel we should at least limit our exposure to poor estimates, even if in a commercial setting we are most definitely not in a position to entirely do away with them.

Definition of Done

My catch phrase could be “Done done” given the amount of times I utter it, however this critical part of Agile working is in no way reflected in the simplistic Sprint Burn Down charts offered by the Commitment Driven approach. Hours can be claimed without question or any measurable means of completion, therefore it’s theoretically possible to get close to the hourly burn down objective for the sprint and still have little or nothing to show for it. As a measure of team progress I don’t believe it is a very accurate or helpful one.

Tangible benefits?

Presuming I do use the Commitment Driven approach and the sprint burn down shows the team is behind on where it needs to be, what is the team realistically meant to do about it? Make the sprint longer? Lasso more resource? Order some pizza and make the team work late? Or perhaps I cut quality? This doesn’t feel very sustainable to me, and has more than a hint of the bad old days! The obvious answer is to sacrifice some scope (as it was either poorly understood or estimated at sprint planning) - but this is exactly what we would have done using the Velocity Driven approach so what have we gained?

Feeling the Pressure 

In going through the proposed sprint backlog and producing detailed (hourly) estimates for tasks, individual’s inherent expertise or roles are likely to surface (e.g. John normally does the UI, lets go with his estimate). It is highly probable that the fact that John has estimated something is likely to create a bias on who undertakes the work during the sprint - i.e. more likely than not, John with work on tasks with a John estimate. This is likely to be even more the case in a mixed ability team where weaker members will inevitably feel pressure from the sprint burn down chart not to impact progress by getting stuck on a task outside their normal area of expertise. This issue is ever present in self organising teams, however I feel with the Commitment Driven approach where everything is estimated to the hour, risks sending out the wrong message.

Risk the Inevitable Estimates v Actuals Comparison

Having an absolute estimates (in hours) will inevitably lead to some bright spark wanting to compare estimate against actuals. You could view this as good or bad (I’m generally inclined towards the latter). If you are up for the Commitment Driven approach, then you should also be supportive of more accurate sprint burn down charts, therefore you will find it hard to push back on this as on face value it “could” legitimately make the team’s estimation better. However, push back I suggest you do, as the level of detail in a User Stories should make reliable estimates a challenge regardless of the approach (unless that is we want to load Stories with significant amount of detail). 

Respect and Trust

In the spirit of creating a sustainable pace, I prefer not to perpetually have conversations around team performance against a set of inherently unreliable estimates. Instead I prefer to give the team the benefit of the doubt that they are working to the best of their ability, and prefer to observe the team to confirm this assumption.

Yes performance is very important and needs to be discussed regularly, but I would prefer to concentrate on conversations around delivering finished units of work, quality and business value first and foremost. The risk of doing otherwise is that we subtly change the team’s behaviour to deliver what we promote (in this case performance) at the likely expense of longer term technical debt.

Commitment != Guarantee

Mike Cohn clearly makes the above point, namely the team is making a commitment not a guarantee to deliver everything in the sprint. However I suspect business stakeholders are unlikely to have read his answer! The very fact he has addressed this point concerns me. Communications to stakeholders need to be crystal clear - either you are guaranteeing something, or you are not. Commitment to me sounds like I should expect something, therefore I think the alternative suggestion of calling this “Capacity” Driven Sprint planning is a good one. 

One thing I’m sure we can agree on, is that no matter what your approach, neither Commitment nor Velocity Driven approaches are guaranteeing (or committing) to deliver a fixed outcome. If you accept this premise (which all Agilist should), then you should question the point of investing additional effort to provide some additional, questionable, level of certainty around delivering the outcome. 

It would be really interest me to see empirical data on both approaches and the % of sprint objectives met in each. As in general, I suspect this data doesn’t exist however.

What’s the point?

In the majority of the situations I find myself in, I struggle to see the value of the Commitment Driven approach. Yes it gives the team another viewpoint on how they are performing (although I think the options to react based on the information is limited), but I believe it comes at a cost in terms of additional effort, complexity and risk (as discussed above). 

Kent Beck states in XP (Extreme Programming Explained) when talking about teams that they “should experiment with just how little measurement they can do and still be efficient”. In addition, one of the four key values of XP is simplicity - “What is the simplest thing that could work?” - something that I think should equally apply to development processes as well as to the end solution. Both suggest to me that it is better to dialup metrics rather than dialling down, and Beck’s advice is a useful counter narrative for becoming too enthusiastic on metrics from the start.

The Alternative - Velocity Driven

If we don’t estimate tasks in hours or perform some other “clever” mathematics to equate points to hourly estimate (which I don’t recommend!), this means we cannot usefully have an hourly based sprint burn down chart. However all is not lost as:

1. You have the scrum board showing the tasks and what state they are in - When using a board (physical or virtual) I’ve never had an issue with people being unaware of targets or performance;
2. If you want a graph, you can plot completed story points (“done done”) against time - whilst this is not as granular as hours, it’s far more aligned to the teams progress and significantly more defendable upon inspection;
3. You have the daily standup - a conversation has many more advantages in promoting situational awareness within teams than a simplistic hourly sprint burn down.

All of the above is zero effort, as you will likely be doing this regardless. I do however accept that a Commitment Driven approach can, in certain circumstances, be a useful tool. For example:

1. The team has an unhealthy penchant for large stories (i.e. a sprint made up of a few large stories which makes tracking progress difficult). Normally a little more time breaking these down into smaller stories resolves this (and indeed many other issues associated with large stories);
2. The business insists on specific bits of functionality being delivered in a specific sprint. Often this is a result of longer sprint lengths being used (as the business “cannot” wait another sprint). I’ve encountered this issue on a couple of occasions, and shorter sprints generally provides the transparency that reduces the problem;
3. This is your first or second sprint, and simply finding your velocity by experimentation doesn’t appeal to you.

Conclusion

During the introduction I included an American football anecdote from Mike Cohn detailing his position on the two approaches. However, Mike’s anecdote on the dangers of Velocity Driven sprint planning is far from compelling to me. I know little about American football, but it doesn’t feel very analogous to a team delivering software. In American football I’m sure the makeup of the team will change from game to game (injuries, transfer etc), the venues and weather conditions may vary each time, and the aim of the game is not to maximise the score line - merely to get more points than your opponents. It is a competitive undertaking unlike delivering working software. If software delivery was like football, then perhaps we could all start a new discipline of agile velocity “betting”??

I do concede that Velocity Driven sprint planning is likely to be a little less accurate, however it requires much less effort than the Commitment Driven approach and significantly simplifies some of the team’s processes. Furthermore, we don’t have to use best estimation practices in one part of the delivery process (i.e. backlog estimation), only to fall back on the worse kind of “off the cuff” estimation in another (i.e. sprint planning, effort to complete), which certainly helps my OCD. For both Velocity and Commitment Driven approaches, the performance levels of the team should be the same, and any issues around tracking progress caused by not estimating in absolute terms can be largely mitigated by driving better agile practices (smaller stories and shorter feedback loops) and using story point based sprint burn downs.

Finally, should a team find value in investing a little more upfront time in planning out a sprint in more detail (perhaps there is a large user story that defies being broken down further), there is nothing preventing them taking up the Commitment Driven approach at the next sprint planning session. Using the Commitment Driven approach as a tool rather than a standard part of the process feels more comfortable to me.

As always in delivery, pragmatism and situational awareness is the key to success. If it works for you - it's good.