Code quality in numbers and how to track it.

Recently I was a part of a team that was working on a massive refactoring and bug fixing project. It was a big legacy beast and it was not surprising that it took quite a few efforts and nerves. A fair part of it was due to initial lack of consistent design approach but then it was mostly because of bad code quality and almost total absence of unit tests. I had another refactoring project a few years ago when it was completed relatively smoothly mostly because of a good number of unit tests in it. Here I saw many times that fixing a bug in one place would broke it in a few other places. Usually it required a few moves forth and back between testers and devs. I realized that we were stuck on things that could be fixed with a few unit tests. To add a unit test would require some efforts of a devs time, but in general it would save time of testers testing it and devs fixing it and then again testers testing a fix. A couple of days instead of just a half an hour to write a unit test. Such a massive waste of money. Then I started thinking how to make it better. We needed somehow to track code quality and see where we are heading to.

As a result of considering unit tests as a way to improve code quality I searched for a definition of code quality. I knew that it was a complex measure but wanted to find out concrete metrics and tools to measure. I found that Visual Studio has a built-in tool to do a static code analysis. Clicking on "Analyse/Calculate code metrics" displays a table with 5 columns of quality metrics and rows broken down by assemblies and types:

I thought that it was a good idea to see the numbers and not just a personal subjective perception. So, the next question was how to track these numbers. Suggesting devs to calculate it before every check-in on their machines would not work. We need a central point which is a CI build, in my case VSTS 2017.

Code coverage can be switched on in a build definition straightaway. But it would be useful to see the progress between builds or for the whole sprint. Then I found a third party component that can be included into a build definition - NDepend. It does a static code analysis using quite a few code quality metrics broken by types or urgency. The most interesting thing is that it has “Trends” tab where various graphs included to reflect a timeline of a state based on a baseline.

It gives a very clear way to see whether we are improving our code quality or not. Also, it has a Quality gates where various quality thresholds can be set to make build failing when some metrics are degrading. The problem with this component is that it is not free.

There is another component - “Check build quality” which is free and currently supports two policies: “Code coverage” and “Warnings count”. Thresholds can be configured to set up a quality gate to force build to fail when code coverage is decreasing or number of warnings is growing comparing to previous build.

Finally, as a result of these investigations I can at least offer a few ideas how to consider code quality and how to track it.

ps. While looking at different components I came across a few more that do a static code analysis but did not have time to investigate. So, the next task for me would be to find out a free alternative to NDepend.