Battle of the Q: Quality vs. Quick
I met with a group of coders last week and there was a discussion about the need for unit testing every possible condition within a function. For example, a function that contains 10 execution paths should contain, at minimum, 10 tests.
I’d argue that it should have at minimum 20, one positive and one negative test per execution path. And if you are going to be really robust, only one positive and one negative test per execution path is probably insufficient, as there are likely dozens of states per execution path.
A pedantic coder could easily come up with 100 tests per function or more.
Controversy alert: I rarely write unit tests. I’ll get to why in just a moment.
Gamma and Beck, two members of the famous Gang of Four (creators of design patterns), came up with jUnit during a flight back in 1997, at least according to legend. By the early 2000s, it had made its way into popular programming communities and I loved the idea.
At Visional Corporation, our code base was written primarily in C++ and Java I used cppUnit jUnit extensively.
I worked on that code base for almost 10 years, looking at the same code every day, so I had plenty of time to think about how to best perfect it.
Over time I moved over to stacks that were easier to deal with. Code became more self documenting. Compile time was eliminated. Change became less expensive (though it’s still more expensive to fix a bug than to spend extra time on the front end to make it work.) I became less reliant on the testing frameworks that, often, didn’t cover the most problematic use-cases anyway.
But that’s not the number one reason I am not heavy into test-driven development.
It’s because there is a force of change than a coder cannot avoid, one you cannot code around. The change of a mind.
For the last several years I’ve worked for startup companies, building systems from the ground up, and my thinking evolved to suit the environments.
You see, tech startup companies begin with an idea for a project. Always, once they see the first working version, or get it in front of a customer, they realize that they missed the mark. They want a do-over.
A startup company doesn’t want hardened code that will weather a decade. They want to see something they can put in front of an investor or potential customer as quickly as possible.
For example, the code base at MachineParty.com, a company that I've been working with since January of 2015, has already undergone three major rewrites. Ideas evolved. Concepts changed. Note that the code base is less than a year old.
While working with startups I learned to write good-enough code. Good-enough code meets these objectives:
- It’s clean
- It’s self-documenting
- It doesn’t repeat itself
- It works
Good-enough coding keeps its eye on the what is to be delivered, with an understanding that, if the code sticks, it can be refactored and improved in future sprints.
Good-enough coding is an understanding that what you write today, no matter how robust and hardened, will likely be replaced next year, if not in six months.
Good-enough coding helps startup business win the race to be the first to market.
Good-enough coding helps project owners realize early when the development group misinterpreted their intentions.
I’m been a fan and practitioner of Agile development since the early 2000s.
Sometimes developers miss that Agile is more than just a way to limit the scope of work for the developers. They need this limit so they can get something real done, and it’s important. But Agile is also about improving the way value is delivered, by providing quick feedback, quick iterations, and the ability to quickly pivot when needed.
Agile attempts to strike a balance between Quick and Quality.
Over-engineering is the process by which coders built today for a future which most often doesn’t really exist. Build for tomorrow and in three months we’ll see if there is still a future for your function.
I can’t leave this topic without touching on Not-good-enough coding.
Not-good-enough coding embarrasses product owners when the site keeps going down.
Not-good-enough coding is full of duplication, which leads to unsustainable, brittle code as various coders refactor the duplicate parts in different ways, introducing complexity that is difficult to remove over time.
Not-good-enough coding causes the developers to hate their jobs, as they struggle to fix the mistakes of former developers who moved on.
In truth, a perfect development team is composed of both Good-enough coders and Pedantic coders. Pedantic developers tend to drag the timeline, but they boost quality. Good-enough developers push the envelope and move the product forward. Together there is a healthy tension that leads to a win.
Like any other element of methodology, unit tests should be used when they improve the product. In the lower layers of a product, especially when the layering corresponds to an organizational hand-off, unit tests add value in some concrete ways that have very little to do with exercising all the code paths. 1, they document the interface. Programmers on the other side of the hand-off can read the unit tests as a go-by for using the module. 2, they provide an entry point for debugging. Having a unit test module that will or can easily be enhanced to reach a particular state in the module helps in reproducing bugs. 3, to the extent that they cover the required functionality, they provide a safety net to make refactoring less risky. 4, attempting to create a unit test suite exposes isolation flaws in the interface contract and/or the implementation. My reason 3 is the closest to the conventional justification but in practice you don't usually have that much coverage where you really need it -- unless you're paying the price to put it everywhere. One way to have it where you need it is to move the responsibility onto the consumer -- if you're going to use my module, you should write unit tests to verify the interface contract terms as you understand / intend to use them. If that policy were used in practice I would be free to refactor or even alter the design of the interface, within the "as-consumed" bounds defined by the union of my consumers' unit tests. On the whole I agree, though, that unit tests have not in practice lived up to the hype.