Data Modelling: Bridging the Gap Between Technical Experts and User Needs
The Context
Analogies are fantastic. They are great at conveying complex ideas or concepts in contexts in which most people can identify and understand.
My colleague, Chris Williams , has a great analogy for a data platform illustrated by an orchestra. I would do it and him a disservice if I tried to reproduce it, so I hope that Chris one day shares it with the world.
Recently, I have been having debates with colleagues and customers about data modelling as there's a belief that data modelling is a technical skill that should remain in the hands of technical people. I disagree - data modelling is a skill focused on asking the right questions.
The Analogous Car
Now, time for an analogy. Say we have two groups of users, a technical group filled with engineers and a non-technical group filled with non-engineers. If we ask each group to design a car, we'll end up with two very different designs. One resembling an F1 car and the other, perhaps, resembling a Honda Civic.
We'll respectively label our groups: Car Engineers and Car Users.
In isolation, the two groups have designed, conceptually, the same thing - a car. But they have approached the design from different angles and different disciplines.
The car engineers have approached the design from attempting to resolve technical problems:
Their focus extends beyond the surface-level goal of winning races, delving into intricate technical considerations, such as keeping the car grounded during high-speed turns.
The Car Users, on the other hand, have approached the design to draw out features that align with their needs and desires. For example:
Recommended by LinkedIn
However, for a car to be built and the requirements identified by users to be realised, you still need the technical expertise of the engineers to make that happen - and to also temper some of the requirements which may be unfeasible or not compatible with other requirements, as they be more applicable to another car, for example a Porsche 911.
The Design
The design of a data model needs to come from users - driving the requirements for and the adoption of the model. To make a data model user centric, start by asking questions about a process in their domain, before digging into the details of the events that occur in the process - as this is what we're trying to model.
Collaborative and iterative data model design methodologies, like Sun Modelling, BEAM, and SunBeam all aim to put the user front and centre of the design. Technical concepts and columns are rarely mentioned, because it all about the process, events, and a design which supports a business.
The Implementation
Once the model has been designed collaboratively, the technical expertise come to the fore for the implementation of the model as well as the translation between business requirements and technical feasibility. It is not a foregone conclusion that everything that is asked for can be delivered, nor is it to be expected that everything can or should be delivered at once.
Back to our car analogy - the car has evolved significantly in the 100+ years since the dawn of the automobile, yet each iteration is still recognisably a car. The same is true of a data model and all analytics products. The design and implementation of analytics products are never final, they are a continuous process of improvement, reflecting the nature of the organisations in which they exist.
The Conclusion
As we've seen, through the use of analogies, that data modelling is a collaborative and iterative experience for all. Neither business requirements or technical expertise are more important than the other, you need both to deliver an effective data model. Nor is the design or implementation of any model static - it's a continuously improving process to reflect the organisation in which it exists.
If you want to find out more about SunBeam, or try out collaborative data modelling with the support of experts, just let me know in the comments or in the messages.
Yes! Breaking down silos and ensuring data accessiblity and analytics adoption is so important. You can build the F1 car but if users don't know how to drive it, it's not the correct solution. Collaboration is the approach that makes the most sense (and delivers the most value).
I don’t think you can get away from technical details. For example deltalake only collects stats on the 1st 32 columns. There’s mountains of these details constantly changing. When designing physical models the use cases matter but so does the technical details and the road map of those details. They matter so much because usability is front and centre of all data use cases. The right Performance is an assumed requirement. Without it everything else is lost. The holistic modelling is intricate detailed and nuanced. It’s basically the usability of data. We talk about quality, modelling, performance… but in the end how usable is it holistically?