Are you measuring to fail?

There’s no doubt that measurements can help us to achieve results whether it’s in a personal or a business sense, but like most things in this world they have a dark side that when left unchecked can destroy worlds! Okay so maybe that’s little dramatic but we do need to be mindful that anything that can be measured can also be gamed.

According to Campbell’s law:

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures, and the more apt it will be to distort and corrupt the social processes it is intended to monitor”

Quantitative measurements are prone to abuse as they typically measure a single aspect that is linked to an incentive. The so-called cobra or rat effect.

The Cobra or Rat Effect

The cobra effect occurs when an attempted solution to problem makes the problem worse, as a type of unintended consequence. The cobra effect originated as an anecdote during the time of British rule of colonial India, when the British Government introduced a bounty for every dead cobra to try and reduce the cobra population. As an unintended consequence people started breeding cobras for income which had the adverse effect of increasing the population. Needless to say the reward program was scrapped.

A similar thing happened in Vietnam under the French rule where a bounty was introduced for every rat killed. No surprise what happened here.

According to Goodhart’s law:

“When a feature of the economy is picked as an indicator of the economy, then it inexorably ceases to function as that indicator because people start to game it”.

Marilyn Strathern phrases Goodhart’s law as “When a measure becomes a target, it ceases to be a good measure.”

When it comes to metric it’s important to;

• Use quantitative and qualitative measures to avoid achieving a single goal at the expense of other equally important goals
• Remember the original goal, review and possibly change the measurement every few months

The Objectives and Key Results (OKR) framework does this well by defining how to achieve the objectives through specific and measurable actions. It also doesn’t hurt that Google had a lot of success with OKRs, which is probably why it’s popular among tech start-ups.

One final thing to keep in mind is that a measurement can be both ‘good’ or ‘bad’ depending on how it’s used. A good example of this is velocity in Scrum

Velocity in Scrum

It’s a useful measurement for a Scrum team to see how their performance is tracking and also for release planning.

It’s a ‘bad’ measurement for management to evaluate a team’s performance or worse still an individual’s performance. The counterproductive results of using velocity for measuring a Scrum team’s performance are;

• The team changes the definition of done to save time resulting in lower quality code
• The team performs the same level of work but clocks a higher velocity
• The team losses motivation and passion and reduces their velocity

Measurements are valuable when used to indicate progress but are dangerous when used in the wrong context, doing more harm than good.