Bridging the communication gap in data science

When I first came up with the idea of writing Analytical Skills for AI and Data Science (ASDS), I was motivated by three quite common experiences I usually had:

1. As Head of Data Science I was constantly trying to find opportunities to collaborate with different business stakeholders across the company. Almost invariably, I found that my stakeholders couldn’t see more value in data other than the creation of timely dashboards.
2. Right around that time I was also teaching a course on Big Data for Managers for MBA students. I had been doing this for a while and, cohort after cohort, I kept finding the same blocker: the apparent inability to see value in data other than the merely descriptive.
3. The third blocker I found was that data scientists in the team generally lacked the vision for finding and creating new proposals for their business counterparts. Since their business stakeholders couldn’t move from the descriptive use (1 and 2 above), this passive role generated the risk of moving the company to a dashboard/reporting low-productivity equilibrium that I believe is quite common across enterprises.

My aim with the book is to help overcome these difficulties by describing a general framework for finding new projects that use as inputs all of the company’s data assets (data, infrastructure and the talented human capital). Non-technical business people benefit by learning the actual potential for engaging in data- and prediction-driven decision-making (the prescriptive ideal); data scientists can also have a more proactive role in coming up with new ideas.

Where is the communication gap?

Generally speaking, data can be used to provide a picture of the current and past state of the business (descriptive stage), as input to create predictive models for the future state of the business (predictive stage), or as inputs for improving our decision making processes (prescriptive stage). This is well captured by standard data maturity models such as the one in the next figure (taken from Figure 1-3 in ASDS).

Figure 1: The three uses for data

What I call the communication gap is the generalized inability for both parts --- non-technical business stakeholders and data scientists --- to move from the descriptive stage to the potentially more valuable predictive and prescriptive siblings. As I said earlier, a likely negative outcome is to take the company to a low-productivity loop that causes a lot of organizational frustration: business stakeholders demand reports from their data scientists, and since they lack the ideas to supply data products with the potential of creating more value, they loop around dashboarding and reporting (Figure 2).

Figure 2: Communication gap creates low-productivity outcome

Overcoming the communication gap: business stakeholders view

With my students and business stakeholders it was always clear that they felt very comfortable living in the descriptive sphere. No matter what we do in the company, we’ve always worked with reports created with spreadsheets like Excel or Google Sheets. 

When challenged, I also found that moving to the predictive stage was everything but natural for them. Except for those with an engineering or an economics background, the prescriptive stage was not even in their menu of options. Why is this the case?

It’s all about expectations

The descriptive use of data is so ingrained in our subconscious that considering something different is generally out of the question. To replace those expectations we need to educate our colleagues on other uses for data. This data- and model- evangelization task is so important that many data executives make it an integral part of their mission. Also, constant support and sponsorship from the executive committee, and the CEO herself, is generally key if we want to run the business in a data- and model-driven way.

The problem with the term “AI” (Artificial Intelligence)

Another problem I’ve found is that the term “AI” generates unattainable expectations. When I was Head of Data Science I had to constantly remind some of my business stakeholders that the AI solution that a specific vendor or consultant had offered was no different from what my team had been trying to develop with them for months. I decided to include the term in the title of ASDS with the conscious purpose of making it synonymous with what data scientists do. Once we take the superhuman aura away from it, we can finally start talking the same language: data- and prediction-driven business decision-making.

Prediction for what?

As I explained in another post, the intermediate standing of the predictive stage in data maturity models such as the one in Figure 1 is anything but obvious. Why do we need prediction? Or even better, how can our business stakeholders be better off from making better predictions.  

Sure, thanks to better (and cheaper) predictions we can move from reaction to predictive action. Think of customer churn: if I can predict which customers will leave tomorrow, I can come up with some retention strategy today. Herein lies the value for prediction for most business stakeholders: data science buys us time to come up with better decisions. Which decisions? Most likely the same we have been using until now, i.e. business as usual (BAU).

Moving away from BAU

But why do we need prediction in the first place? Uncertainty is the raison d'etre for having prediction models. In ASDS I claim that identifying the relevant uncertainty in a problem is an analytical skill that is easily learned, and quickly helps overcome part of the communication gap.

Some years ago I was trying to convince my business stakeholder that we could use the data science toolkit to help him reduce salesforce fraud in our company and he just couldn’t see how this was possible. After several weeks of engaging in the same dead-end conversation, I finally decided to challenge him: “the main problem you have is that you don’t know which customers are real and which are fraudulent”. This was the underlying uncertainty for his business problem. “What courses of action would you take if you knew exactly where the fraud is?” This very simple conversation allowed us to come up with a full prescriptive plan to reduce fraud.  

I've found this technique quite revealing: start by solving your business problem without uncertainty. What is the best you can do? Now use machine learning and answer the same question with uncertainty.  

Overcoming the communication gap: data scientists view

The communication gap from the perspective of a data scientist has a different flavor: the main problem is that we need to take them closer to the business. The general framework I propose in ASDS is to equate “business” to “business decision-making”. By doing this we can take advantage of the rich and well-known structure that decisions have (Figure 3, taken from ASDS).

Figure 3: the structure of a typical decision

My general approach towards data science --- and the one I describe in ASDS --- is centered in business decisions. What decisions are your business stakeholders trying to make? What business objective are they trying to attain? Is that the right objective? What are the levers they can pull? Are there other levers we can experiment with? What are the consequences from pulling those levers? What are the causal mechanisms that tie actions/levers to consequences? What is the underlying uncertainty? Can you use machine learning to partly overcome this underlying uncertainty?  

Adopting this framework allows data scientists to systematically think about the business which naturally helps to overcome the communication gap. A nice advantage is that since this is a general methodology, it's easily transferable across business problems within the same company, or across companies and sectors. This approach involves learning some new analytical skills, but I’ll leave that to a future post.