Predictability and productivity

Working for companies with OKRs or quarterly roadmap goals, I’ve been noticing an interesting phenomenon. Features invariably landed towards the end of the quarter (if the goal was hit at all), but practically never mid-quarter. This happened without fail across very different organizations and products. As I gave it some thought, my main observation was that all groups measured success through a quarterly feature delivery lens.  

When an organization measures success by its ability to hit particular dates, it sends a clear message: predictability is king. There is nothing wrong with that, but it drives certain behaviours that stem from inherent inaccuracy in R&D estimation (or any estimation in general). There’s enough written on this topic, so I’ll just recap a few known points: 

1. Developer estimation is inherently optimistic, even when requirements are well understood. 
2. In a date-driven organization operating on a quarterly schedule, teams are often under crunch to deliver and rarely have the time to analyze requirements and solution in depth, and commit to goals with limited data. There is a negative feedback loop. 
3. In Lean organizations, requirements are fluid; they change as we learn from our customers, which may also shift the dates post-commitment. 
4. There is often a business pressure to make a certain goal happen, which creates downward pressure on estimates, even when they are accurate. 

Experienced teams are aware of that, and adjust their behaviours accordingly. They may: 

• Pad their estimates to budget for unknowns in points (1), (2) and (3), and to anticipate push-back from point (4). Scotty’s principle is one typical implementation.  
• Fit the requirement to the date. So, if the date is at risk, the requirement is changed to fit the date – for example, a feature would get released to a smaller set of customers, UX would be simplified etc. 
• Cut down on work that does not have a similarly visible goal – be it tech debt reduction, or grooming requirements that might come in future quarters. 

Going back to the original observation, they’ll also:

• Deliver features towards the end of the quarter, because delivering sooner would make a hard task even harder while hitting exactly the same goal. 

This answers the initial question on why features tend to come towards the end of the quarter, but leads to a more interesting question: 

Is there a trade-off between predictability, quality and productivity? In other words, if we removed all time-bound milestones from the delivery process, would we produce software faster and/or better? 

In my opinion, most of the time the answer is “no”, and my favourite example would be the hackathon. Hackathons are compressed to a couple of days, and this tight constraint is what triggers the burst of productivity: more often than not yielding fascinating results. 

While a hackathon is arguably an atypical example, it shows that productivity and predictability do not have an inherent conflict. But then, when we go back to the quarterly and monthly scale, how do we sidestep all the side-effects covered in the points above? 

The one key difference between hackathons and committed roadmap is psychological safety. Missing a roadmap commitment (unlike missing a hackathon goal) may have implications: be it in terms of team performance perception, customer expectations and so forth. When dates rule the kingdom, they may push aside other aspects that matter – such as feature’s fit to customer needs, quality, and maintainability. 

All of this brings us to the subject of culture. Culture is about what is being rewarded and discouraged in an organization. If an organization rewards only hitting dates, it risks optimizing for the wrong goal – and driving the behaviours above: estimate padding, modifying requirements to fit dates, accumulating tech debt, and completing the feature on the tail end of the target time period. 

If, on the contrary, real business needs get rewarded – adoption, productivity, and quick reaction to customer feedback, such behaviours are less likely to happen.  

Let me contrast that through a few examples: 

Date-driven delivery goal: “Deploy the new Acme widget by the end of Q1”

Adoption-driven goal: “Have at least Y customers interact with the new Acme widget in Q1” 

Date-driven delivery goal: “Deploy at least 3 new features in website search in Q1”

Adoption-driven goal: “Improve successful interactions in website search in Q1 by 20%, where successful interactions mean a click on the first page of results” 

Taking that last example, both goals can trigger the same actions, but the first goal is more prescriptive and likely to cause unwanted side effects that don’t help the desired business aim. 

See more here on the same topic.

So, in summary, there’s nothing wrong with focusing on dates, and measuring progress against them. However, they are means to an end, and that end is driving revenue, reacting to customer needs and differentiating from competition. As long as we measure and reward what matters, we are less likely to hit unwanted side effects.