Thinking vs. Coding

When I was growing up as a programmer, it was the time of object-oriented-everything. You were first supposed to do object-oriented analysis (where you you tried to understand the part of the system/domain you were about to implement), then there was object-oriented design (where you'd come up with the program structures that represent what you'd analysed in a way that had the right -ilities and degrees of freedom) and then, finally, you'd grab your IDE and write the code.

This rather strict sequencing is crazy, especially when, as it was kinda normal at the time, it was combined with a waterfall process; the analysis and design phases could last weeks. There were nice german words for the documents you were supposed to write, the Lastenheft and the Pflichtenheft. And the processes of the day, eg., the Rational Unified Process had dozens of dozens of documents you were supposed to produce before you were supposed to start coding. I remember a project at the end of the 90s where the architect expressed his delight about the fact that the RUP was "customizable" for the context it was used in, and that we only had to write 80 of these prescribed documents (no exaggeration here, it was 80). Luckily, hardly anybody did it this way.

Today we're living in a world of agile where any kind of document has a bad rep. Remember, the Agile Manifesto states that we value "working software over comprehensive documentation" and "responding to change over following a plan".

This is often (mis-)interpreted as saying that we should not think about the domain or problem before we start coding; just build it, write tests and then see if it's useful. The agile pioneers didn't mean it that way. They wanted to prevent exactly these weeks and months long of paper production phases. This is because they realized that the stakeholders often don't really know what they want until we start building early versions of a system, that requirements change over time (even during the time it takes to build a system), that requirements documents cannot be tested or otherwise verified for consistency (and so they are often incomplete, inconsistent and inscrutable), and that even developers and requirements analysis cannot build layers and layers of mental models without first implementing something and then continue building mental models on top of that executable system.

But here is the problem: many people today really do understand the Agile "prescriptions" as "let's talk about what the customer needs for a few minutes, and then let's build something, we can always iterate and change". This might work alright for simple systems. But if you're trying to represent a complex domain in software -- for example as a DSL, but also in other shapes -- you have to spend some time to try and understand

• what are the core abstractions of the domain and how do they relate
• which variability and special cases exist in these abstractions
• which parts of what people tell you as "requirements" are really relevant for the new system and which are just historic accidents and should be ignored or replaced with something better
• in which form should we let users interact with the domain (syntax, etc.)

These "analysis sessions" can last for hours, usually include software (language) people and domain experts, and yes, sometimes we even produce a couple of pages of written material.

The key to success of such sessions is three things:

• There must be some people in the session with significant knowledge about the domain for which we want to define the system to answer questions about how the to-be-built system should work. If you are building a system that replaces an existing one, they also have to be able to distinguish how the current system works versus how a new system is supposed to work.
• Everybody in the group must be able to abstract. You will be conceiving mental models of how a future system should work. If participants can't do that, this will be a problem.
• People must also be able to communicate precisely and clearly both about the domain, and about how the currently-designed system works based on the current state of the mental models you are building.

A final very important ingredient for these "thinking sessions" is that you recognize when you have to stop thinking and start coding. The pyramid of mental models can only become so high before even the best abstractionists run out of steam or when feedback from future users is needed. At this point stop the session, build a prototype, and learn from it. Go back thinking later, if needed.