"Information Technology" is about *Information* and NOT about *Technology*
Introduction
I come across a lot of discussions around the (presumed) death of data/information modeling/architecture/management/whatever in the current era of "Big Data" and "Analytics". Professionals and customers alike are wondering if and how we need to look at these subjects now the focus is shifting towards pure data valorization(monetization) technologies, at the expense of the more 'traditional' data modeling skills and activities.
Information first
Most professionals and customers don't realize that from a customer's perspective technology should not matter, only functionality does. Data abstraction and organization is there to contain and rationalize away from technology choices. And we humans are always doing this, even if it is only in our own mind. All data modeling and data organization aspects are there to make this explicit and manageable to abstract away from implementation details.
Data/Information modeling
All 'data modeling' done as part of requirements engineering, or as part of system design should be seem as technology abstraction mechanism's which are important techniques to facilitate data management in it's many aspects, from specification, governance, data quality to data logistics. As long as control over information is needed, (good) data modeling has it's place.
Data and technology
For many reasons technology oriented forces are constantly ignoring, downplaying or opposing this technology agnostic approach to data. There are few whose interest align with this abstract notion of data management. This creates constant pressure on data management/modeling/architecture that is NOT technology focused. Indeed only those benefiting from good data management have an interest. For a lot of software or service providers on the data side of things this does not hold. In fact, it often hinders them in their business. Alas, customers usually presume their technology partners will also provide them with good data management, failing to understand that this is exactly what their partners are NOT providing because it's essentially not technology.
Technology pressure and data modeling.
To withstand the current pressure from technology and the need for monetization we need strong(er) data -modeling and -management skills. These do not materialize out of thin air. The amount of time and money invested in technology and it's valorization are immense. The amount of time and money spent on (good) data -modeling and -management is (quite) small in comparison. The current sentiment is that there is not only no money made in these areas of expertise, but that they are even detrimental for data valozrization. This is the trend that needs to be reversed, and this is not easy at all. For an average data modeler this is a tall order indeed. As a profession we need to own up, break with the past and put a brake on the T in IT
Data as Usual?
Does this mean we go back to business as usual? No, because the increased focus on data valorization indeed should trigger us to do better data modeling and data management. In the light of increased technical diversity we should strengthen our data organization and modeling technology, skills and knowledge to better facilitate the increase in data valorization. Alas, where technology is pushed mercilessly by large organization's, better data modeling and management skills are hard to come by and usually are far more dependent on the commitment of the individual. Now even more so the say 20 years ago.
But it's also and imbalance between the forces of structuring and organization and the forces of usage and valorization. And an imbalance to these forces, which known to several oriental philosophies as the yin vs yang approach, can make the organism ill. We saw this partially in the totally unexpected financial crisis and the current response by financial supervisors. They focus more and more on actual data management of financial and risk data directly instead of the financial corporations themselves.
"If you see these same patters in your environment, realize that data modeling is of interest to data management as one of it's prime deliverables and steering aids, and understanding the balance between data management and valozrization is of the essence. Stay tuned for a follow up post on the the art of balance in data management"
ABOUT THE AUTHOR:
Martijn Evers is Chief information architect at and co-founder of I-Refact, delivering top data engineers and architects to high profile organizations helping them to managing their data effectively. He's an expert facilitator and researcher on a large range of data modeling topics and is regarded as one of the best data architects around.
Agree with everything in this article. I think the author needs to be careful with opposing "technology agnostic" forces, which can be interpreted as "vendor agnostic". I think one of the reasons for this division between technology and good data management and modeling practices is because IT strategists have been sold the idea by hardware and software vendors that their solutions will take care of principal data modeling and thus reduce the number of resources necessary to implement a solution. Many times this proves to be fatal to a properly implemented architecture as the realities versus the perspectives play out in a live production environment, after the vendor and customer have already signed the contract. Taking a "vendor agnostic" approach to data modeling will ensure clients have the knowledge and governance to make sure that their mandates in regards to data are adhered to by a vendor.
In Chapter 5 of his 1978 book, Data and Reality, William Kent made this statement: "We want to define our basic information constructs in real world terms; the implementation in data processing mechanisms comes after we model the enterprise, not before." We should adopt the view that Kent's reference to "information in real world terms" was not just about the primacy of information in the wild but about the connection between it and the language (terms) we use to encapsulate it. By extension (and intention, to drag set theory into the mix) the technologies we use to manipulate information should at least recognize that it has a natural 'real world' dimensionality that is universally definable.
When I was half my current age I used to wonder why the grumpy cats in the office acted that way. Now I'm the grumpy cat. I was asked yesterday if I've worked with Hadoop. I said no, but I've worked with KSDSs before relational and that a lot of old stuff just gets recycled with new names. As the young ones usually are, my colleague seemed unconvinced.
Agree - it's about people and information. Technology only enables.
While developing high quality data models for applications is a formidable task and an excellent practice that yields obvious benefits, the next-level challenge is aligning these disparate data models to create an enterprise data-view that provides significant synergy in terms of data exchange/integration/reusability. That is where data modelling evolves into (meta)data architecture. Very few organisations master this capability, and few (if any!) tools support it. I'm looking forward to your follow-up post, Martijn.