Getting Business Value from Big Data

When discussing big data with customers, I often run into confusion of our own making before we get to the important questions of how to actually use these technologies to drive business value. Whether its the “4 Vs”, Hadoop vs. Relational, or structured vs. unstructured, our industry has a remarkable capacity to create hype we then have to dig through to help our customers make sense of it all. One of our buzz phrases that I do like to hear from the customer is “schema on read vs. schema on write”. This one gets less air time, but opens the door to connect the technology to something every single customer wrestles with and solves a daily source of friction between their business and their IT teams.

For many decades we have managed the information life-cycle for decision support systems much like we have for core transaction systems. You documented the business requirement, created a data model to support it, built a database schema and then loaded the data into the database. From there you repeated the cycle until the business had what they wanted. This process is time-consuming and expensive, but for operational requirements it fits the bill. You know the business process, you know the expected answer and the goal is to get it right and to run it over and over without failure. 

But what about the research and discovery process that proceeds the recognition that something could be valuable to the operation of the business? Here the information life-cycle needs to be different and the process described above is actually an obstacle. Every customer has the problem where teams bypass this process by necessity and create their own solutions, or “shadow IT” groups. They need a faster, less expensive process that gives them the ability to iterate and discover value with much less dependence on IT. IT needs the ability to meet this need while at the same time meeting the governance and compliance requirements of the larger enterprise, especially in highly regulated industries like financial services and health care. 

It is here that big data may have its greatest potential for the broadest group of customers. Big Data provides a very efficient and cost effective way for IT to focus on what it does best; provisioning data to the business and enforcing the appropriate access and use of that data, while freeing the business to do what it does best; converting that data into information they can use to improve their business. In the research and discovery phase of the information life-cycle, this means an environment where there is very little technical overhead and rapid trial and error iterations are the norm. 

The value the “schema on read” aspect of big data can bring to our customer is often drowned out by all the hype we create about what big data is and isn’t. In the day to day running of a business, showing how big data can be used to store and provision raw data to the business for a research and discovery “sandbox” can be a game changer. With the powerful self-service analytic and discovery tools in the market today, the business now has a place to innovate quickly to generate value from their data capital, while IT has an environment where they can apply different levels of governance and controls much more efficiently and effectively for the enterprise. Big Data delivers a “win/win” solution to the information life-cycle problem every business faces. This is rare in my experience and we need to do more to show this value to our customers.