Metrics for Developer Productivity

Companies love numbers. They love to boast about their numbers and are constantly developing ways to measure even the most mundane things ranging from daily employee coffee intake to smoke breaks. But the one thing companies have tried and failed to measure is developer productivity. 

Many have tried, but few have succeeded in creating the right metrics for developer productivity. It all boils down to the creative and abstract nature of the field. Non-technical stakeholders love to use this refrain:

You cannot manage something if you can’t measure it.

But how accurate is this when it comes to developer productivity?

Attempts have been made to measure developer productivity in terms of number of lines of code, number of commits per day or even story points. But anyone who is actually a developer knows the pitfalls of measuring productivity using these metrics. 

For example, Number of Lines of Code can easily be achieved since IDEs generate a lot of boilerplate code, therefore writing more lines of code doesn’t always mean it is the most optimal implementation or maintainable code. This dilemma is further complicated when you factor in the inclusion of comments.

Another pitfall is that the Agile development process introduced the concept of story points to estimate relative complexity. The Agile manifesto did not realise that this would be used by managers for measuring developer productivity. Unfortunately, managers get vilified for using these metrics, all they were looking for is a metric to measure progress in order to raise red flags to management and ended up using this out of lack of options.

So what is the solution?

Here’s my take on this!

The importance of metrics

Measuring productivity isn’t as simple as measuring something concrete like yearly rainfall or office expenditure. Productivity is a very abstract and individual concept that has many (often unquantifiable) variables. Developer productivity is affected by a number of factors, namely:

• Motivation levels,
• Capabilities,
• Type of mentor,
• Team and ecosystem structure.
• Complexity of feature being built
• Integration and dependency with other components

So why is it so necessary to have a system to measure developer productivity in spite of the difficulties in creating concrete metrics?, to simply put

Developer productivity should be used as a tool to improve an individual and their skill-set rather than comparing people. 

Companies are all about optimising the cost-benefit ratios and one way to do this is by weeding out workers who don’t meet productivity standards. But how do you differentiate between an unproductive employee and a productive one if common techniques like quantifying the number of lines of code or story points don’t work ?

By creating the right metrics.

Choosing the right metrics

A problem as complex as measuring developer productivity requires creative solutions. I like to use the example of my weight loss journey when proposing ways to measure developer productivity. Weight loss, like developer productivity, is a complex problem with multiple variables involved. When measuring weight loss, you are not just looking at overall reduction on the weighing scale. You are also factoring in for variables that impact it, such as your body composition, amount of salt intake, duration of sleep, stress and motivation. It is a similar case with measuring productivity. The parts are as important as the whole. 

Photo by Brett Jordan on Unsplash

With my weight loss journey, I learnt to not only document daily measurements on the weighing scale, I also made it a point to make a note of other factors affecting my weight. Some key metrics which I used were:

• Weight - this was an obvious one. It was measured every morning on an empty stomach as soon as I woke up. The key here was to measure my weight at a time when external factors such as food intake did not affect the measurement.
• Food: Calories In, Grams of Fat, Grams of Protein, Grams of Carbs, Salt intake, and Water intake
• Activity: Number of daily steps measured using fitbit
• Physiological state:

1. Was I feeling hungry all the time?
2. How was my sleep pattern?
3. My mental state and energy levels.

Everything was painstakingly recorded on an excel spreadsheet. This data allowed me to tweak my diet and exercise plan according to my weekly and monthly goals. A key benefit of doing this was, even on days when there seemed no progress, my weight loss spreadsheet proved otherwise. The slow but positive progress motivated me to stick to my schedule. 

You’re probably wondering what is the purpose of this example. The point I am trying to make is, you can’t measure progress with just one criteria. That will not give you a whole and true picture of what your developer is accomplishing.

How do you measure developer productivity?

The question that arises is how then do you measure developer productivity in a fair way that can’t be gamed?

The first thing is, as a manager you need to sit down with your developer and work out an action plan. Easier said than done, of course. The most important factor here is that the developer or engineer needs to understand and acknowledge that their growth lies in coming up with an actionable plan to measure their output so that it becomes a quantitative discussion instead of qualitative one. 

In my experience, I have found this combination of multiple metrics to be ideal in measuring a software engineer’s output. 

• Number of bugs filed against the feature: includes but is not limited to bugs by peers, QA and bugs that showed up in production. 
• Number of code review comments left on their pull request or by the person commenting
• Impact on overall test coverage based on the code contributed (did the change improve or decrease over test coverage)
• Source code maintainability metrics (from tools like code climate or sonar)
• Estimated time vs actual time taken to complete a story
• Number of repetitive comments across Pull Requests.

Using these metrics provides team managers with a way to quantify the productivity levels and improvement in skillset of their team’s software developers. Perhaps now you may be able to find the slacker from the powerhouse!