Democratizing the future of analytics

TL;DR highlights 

• Data analysis should NOT only be the domain of BI and Data Scientist. 
• Analytics is NOT only about providing answers to complicated questions. 
• Analytics SHOULD also be about empowring other job functions to ask questions about their context. 
• Analytics job functions SHOULD focus on creating tools for people to explore data rather than producing complete reports.

Empowerment instead of reporting

How do we communicate data-driven insights most effectively to people without a background in analytics? The question comes up again and again when working within the field of data science. We want to present our insights in an intuitive and relatable way that heightens our audience understanding of a given situation and enables them to make better decisions. However, the way we collect, clean, model and present the data has an enormous impact on how a situation is perceived. The perception of a situation translates directly into how desirable outcomes are understood and therefore the decision space that opens up.

Preparing and presenting analytics is therefor a huge obligation within any organization. Explaining the data, the models and the story these tells to any audience requires careful consideration so that managers and domain experts can make the right decisions. Often the role of the analyst is seen as a technical position, but in reality, there is a great deal of explanation and narrative building involved. However, the narrative part of analytics is hidden in layers of data wrangling. Unfortunately, the many technical decision that eventually frames our data-driven decision space tends to alienate other functions within the organization. Managers and specialists will often ask for the abbreviated, single powerpoint slide version of in-depth analysis or bury the report on their desk as something to get back to when there is time. What goes missing here is one of the most vital parts of analytics - domain knowledge. Organizations should always prioritize that managers and experts should be brought into the process much earlier and encouraged to help explore the data. Why is this customer an outlier? How come these patients are doing significantly better than the rest? Why do we see a spike of interest for our content at this particular time? Are we asking the right people about this pressing question or do we need to expand our data? These are all questions that require the input from many different disciplines within an organization besides the data scientist. 

I would venture a guess and say many of you can see the soundness of my argument but would also point out that most organizations do not have the resources to have every function participate in hour-long sessions discussing the validity and relevance of an analytics project.

I agree that more meetings do sound like torture, and that is why I think the next generation of analytic tools should not only focus on creating more powerful tools for the data scientist but also provide other parts of the organization with ways to explore the data based on their ideas, intuitions, and interest. Do not drag more people into meetings, but instead make the tools that empower them in their context.

Taking the time and effort to make the data analytics inclusive increases not only the number of insights but also the overall quality of the insights from data-based analytics. The next generation of tools needs to enable the data scientist to communicate contexts and present interpretations that will allow decision-makers to make better decisions but also let people without a technical background to do deep dives. 

This kind of exploration could potentially be supported by allowing readers to choose between different representations and attributes of the data, so the narrative fits their particular context. There are many ways to do this but consider allowing others in the organization, besides the data scientist or business intelligence people, to manipulate how they want to represent that data, so it is relevant for their context. For example, it is vital that the sales manager can explore the output of a customer satisfaction survey and combine variables and connections to get an understanding of the customer portfolio that is hard to construct without the direct, working knowledge of the customer portfolio. The role of the data scientist should be facilitating the exploration of data rather than deliver the full explanation. 

It is vital the different domains, from sales, production and marketing and IT all have the opportunity to make the data work for their context and help them answer their questions about the past, present, and future. Critical situations in any organization will affect many different job functions but in different ways. A decrease in sales will require adjustments from both sales and marketing, but the problem might be related to production or software services. In these cases, the situation involves analysis and insight on multiple business functions, but the different job functions have to be able to adapt it to their needs. Even though this requires many more job functions within an organization to increase their knowledge of how to read and play with graphs and charts, this is very much desirable to the current state of analytics where it is the task of the few to create and interpret for the many. This centralization of analytics results in some functions acting on analytics collected and designed just for them while other job functions are starved for practical, actionable insights. Let us instead move towards a situation where we create analytics with versatility and usability in mind. 

Build for exploration not explanation.

At Web Summit this year I heard a story of Chief Data Officer who had implemented a large scale analytics project within an organization. Every manager had a dashboard filled with relevant information, and they had been ordered to check the dashboard every day. After some time the CDO took a walk around the company to see how the managers were using the dashboard. He knew they were logging in, but it was hard to see any real changes despite all the insight the managers now had available. What he saw was that the managers had ordered their secretaries top logging to the dashboard and leaving it open before closing it again. This example highlights how many people in organizations end up dealing with analytics precisely because they do not feel ownership over the analysis or empowered by the results. A way to create ownership and empowerment is by creating tools for exploration and not just explanation. We still need an explanation from data science or BI experts, but at the moment we are far from providing useful tools for people to explore data by themselves.

I think it would be beneficial to create analytics with the ability of others to change the perspective of the analysis along three dimensions: 

1. Allowing others in the organization to change the sequence or the order of how the information is presented. 
2. Making relevant data encodings and transformations available to play with for others in the organization. 
3. Highlighting different parts of the narrative by making smaller pieces or subsets the focus of the analysis. 

I think this approach is not only desirable but also necessary. Everyone needs to be more informed about how data-based insights are created because most of us work within fields that are going to increasingly impact by these analytics or predictive approaches either as part of our job or as something that is going to dictate how the organizations we work in operate. 

If you found this article interesting, please give it a thumbs-up and feel free to follow. I publish articles on a regular basis about agile, product development, and applied data analytics.