Predictable Analytics, Unpredictable Outcomes
Big Data: Why Most Companies Get it Wrong
Data-driven decision-making is the pathway to competitive advantage in the modern data economy, and companies of all sizes are looking to data for answers to critical business questions. Many companies mistakenly approach big data from a technology standpoint. The companies that succeed are those that start by first identifying a pressing business problem or challenge and then use technology as an enabler for solving that.
Companies also struggle with trying to buy and tie each piece of the big data and analytics stack like the data ingestion tools, Hadoop cluster software and platforms, statistical platforms like SAS and Statistica and finally, data architecting and data sciences services.
Dell conducted a big data and advanced analytics adoption survey and the results were very interesting; close to 71% of firms mentioned they are using, or planning to use, analytics in everyday decision-making. However, 60% mentioned that they were medium or lower on their level of analytics maturity based on a certain defined set of parameters. Therefore, it is fair to say that while there is a lot of aspiration, the maturity is low, and we have just seen the tip of the iceberg for big data and analytics adoption.
There are five big challenges and impediments towards achieving mature analytics, in my view.
First, as stated earlier, some of the companies put technology ahead of business, which is not the right approach. We often have to move the conversation with our customers from “Hey, we have all this data. What analytics can we achieve using x, y, z tools?” to “Hey, these are my most pressing business problems; how can we solve this using internal and external data?” Then we determine what data we need and what tools are most suited for the purpose. Sometimes, one has to cut through the hype of big data and explain it’s not about big data but about “All data, big and small” that is relevant to solving a business problem. We also observed that companies who take a business-first approach are likely to have a higher ROI on their analytics projects.
Second, a lot of our customers overestimate both the quality and nature of the data that they have. Quality is about easy access to data sets that are complete and accurate, and ensuring that there is one version of truth. A lot of our customers struggle with this because of the legacy systems, multiple data silos in the organization and lack of data governance and stewardship.
Third, it is very important to prioritize and assess feasibility of what can be done to avoid frustration. There is a lot that can be done, but the focus should be on high-impact business problems that can be solved with the data that an organization has easy access to. This also helps in optimizing investments and ensuring high ROI. As an example, with clear priorities, one would create data lakes and data infrastructures in a phased manner, suited for solving specific high-priority business problems, and not “data dump yards” that lack direction and are a struggle to run and maintain.
Fourth, we need to ensure that analytics is built on a solid security platform, and we tread the fine line of privacy cautiously. It is important to provide customized offers based on customers’ unique needs and wants, but if a company is perceived to be encroaching into the privacy of a customer doing that, then that becomes counter-productive.
And finally, I always tell my customers, “Analytics is a means to the end and not the end itself.” It is meant to solve a problem to achieve favorable business outcomes. Sometimes, we see companies get carried away with the hype of big data and jumping into it without doing a lot of diligence in terms of understanding how is it relevant to them.
To summarize, the following is key to ensure we overcome challenges in the analytics journey:
- Business First: Put business ahead of technology
- Data sufficiency and veracity: prepare a solid foundation of high-quality data ensuring veracity
- Don’t spray and pray: Prioritize, start small and plan big. Focus on both execution and adoption
- Be safe: secure your data, customers and the company’s reputation
With big data at its tipping point – with its three dimensions of volume, variety and velocity growing exponentially – the future presents an unprecedented opportunity to leverage the benefits of big data analytics for businesses. The choice is ours – do we prepare ourselves to harness this energy or get flooded by the “deluge?” And it’s now or never!
Great post Prasad Thrikutam and thanks for sharing it! I'm seeing the same things you are in the supply chain analytics realm. There’s a huge amount of data out there which companies can get their hands on and the problem is that most need to advance their people, processes and technology systems prior to being able to start using it efficiently to truly influence the profitability of their organizations.
A few other points based on my experience 1. An analysis of org culture is required before embarking on a large scale analytics driven transformation program, data being a political subject most of the times what leads to failure is not technology or intent, its people not aligning to deliver the goal. So business needs to own the data and set strong direction. A Data Driven culture is not so common in many traditional industries and often times very hard to implement. 2. Even after putting business first many organizations think there is a silver bullet solution for all their advanced analytics problems. On the contrary it is a very hard problem to solve and each situation demands a slightly different approach. For example predicting the failure of a critical component in an airplane VS predicting number of safety incidents in a warehouse vs predicting behavior of identical sensors under different environment conditions. Business needs to be educated of the costs and investments associated with analyzing and delivering value at an true enterprise scale. Buying 10 tools is not the solution, assembling the right team to address the top 10 business priorities is key and composition of these teams can potentially be very different. 3. Many organizations plan for doing analysis once, don't have a clue of how to operationalize and deploy an analytics model, monitor and improve it on a continuous basis. A holistic view point is required analyzing the end goals and IT strategy should align to deliver the same. Thanks!
Spot on!