Predicting Successful Predictive Analytics Projects

The number of conversations I’ve had with customers and prospects on implementing predictive analytics has more than quadrupled in the last two months. It has been interesting to learn their stories and what brought them to this point. The two predominant reasons these organizations want to take this step are: 1) They feel they can improve on their decision making ability which will 2) Reduce costs or increase revenues and improve their profitability. The business issues they want to improve on covers patient being readmitted back into the hospital within 30 days of being discharged for the same reason, risk analysis on financial transactions, student retention and persistence, fraudulent activities, likelihood of crime by crime types and locations, product and machine failures in the manufacturing process, inventory management, and freshman recruiting and enrollment processes for higher education. There are many more business initiatives that organizations can improve their processes and decision making. What was alarming to me from the organizations I spoke with in the last two months is roughly 40% of them had unsuccessful predictive analytics projects within the last six months. I’m going to share the most common reasons why predictive analytics projects fail. I applaud these organizations for their introspective nature and wanting to learn from their mistakes. Predictive analytics can provide tremendous value within so many applications across all verticals. The financial benefits can be incredible. The financial benefits are a distant second in the public safety area where the value is measured by saving lives. Lives of officers and citizens. The main reasons why predictive analytics projects fail are:

Not focusing on a specific business initiative that predictive analytics can enhance,

Assuming the data doesn’t have integrity/completeness issues,

Spending too much time evaluating models.     

Not Focusing

Focusing on a specific business initiative reduces the chance of “analysis paralysis,” where effort is wasted on trying to fit the analysis findings to an undefined objective. Predictive analytics is most effective when it is used to identify expected cases. For example, customers are scored for risk of churn, to predict who is most likely to leave for one of your competitors. The expected behavior is known, but determining who is most likely to engage in a particular behavior requires predictive analytics to identify specific patterns. Though this benefit is substantial, most organizations are also trying to discover something critical that they don’t already know. Many fail in this endeavor because they begin building their predictive applications with somewhat loose goals in mind. They try various models, or alter the underlying business questions over and over again.

Assuming No Data Challenges

Raw information must be gathered from various sources across the enterprise and compiled in a final data set that is fed to the predictive model. Many companies fail to properly select, cleanse, and enhance the data to make it truly ready. Others are totally unaware of how complete or accurate their information is. They think the data is accurate and reliable but in reality it’s not. GIGO is what happens when the information used in predictive modeling lacks integrity; GI (Garbage In) GO (Garbage Out). If the quality or completeness of the data used is poor, the accuracy of the results will be as well.

Spending Too Much Time Finding the Perfect Model

Predictive models must be evaluated to determine how accurately they predict patterns. Accuracy comes at a cost, and companies must decide how precise they need their models. Is 70% good enough or do results need to be at least 90% correct? You should benchmark the accuracy of the existing process to determine the starting point for a predictive model’s performance.  If the current approach to predicting customer churn is 70 percent accurate, the new model being developed should exceed that. Companies sometimes tend to over-evaluate. They add new variables to the models with hope of creating the perfect model. They test and retest the models, spending tremendous amounts of time making continuous refinements because they are not quite perfect. This delays deployment, and prevents the organization from recognizing the substantial advantages that predictive analytics can offer. There is a tradeoff between time to market, usefulness, and accuracy. No model will ever be perfect but accuracy is key. The incremental increase in accuracy provides the business justification for calculating a return on investment and determining how many more lives can be saved.