Pitfalls of Algorithmic Decision Making

While browsing through my Feedly timeline during the weekend my attention was drown by a blog written by Jitendra Subramanyam of Gartner. The title, Pitfalls of Algorithmic Decisions and how to handle them, made me curious. As I expected, the #algorithms addressed in the article were machine learning algorithms. Subramanyam describes how they are used to automate decisions such as a medical diagnosis, a welfare eligibility assessment or a recruitment decision. The blog indicates that people’s confidence in algorithmic decisions is declining due to incidents of biases and perpetuation of discrimination by these algorithms. They are “weapons of math destruction” as Cathy O’Neil describes in her book. As a result more stringent regulations are proposed by policy makers to protect those affected by these decisions.

You can question if the algorithms Subramanyam mentions really make decisions. They typically make simple judgements, comparing a model outcome with some predefined threshold. The algorithms are the result of applying machine learning to data to create a prediction model for the phenomenon of interest. Usually a supervised learning algorithm is used to create the model. With the resulting prediction model, a score is calculated using the data of, for example, a job applicant. If the score is beyond some threshold the applicant is accepted for the job. To me this sounds more like an automation of judgement instead of decision making, and a very bad way of doing it. All depends on the threshold in this “decision” algorithm, does this do the decision right? It’s a much to simple approach to decision making, especially when the decision is impactful. Can all relevant information be captured in this single threshold value? How can we trust the data used to determine the threshold and verify if it is still accurate and relevant for, in this case, the applicant being evaluated? Next to that, how certain are we about the predicted score? Usually machine learning algorithms don’t provide error bars, making it difficult to verify the quality of the predicted value.

All kinds of comment can be made on the way machine learning models are created and used in automated judgements and much can be done about it to improve it. Even if these models improve, we are still far away from practical application of these algorithms in decision making. Reason is that these algorithms only focus on a single decision, while in practice decisions are usually linked to each other. Let me give an example. Suppose you run a chain of retail stores and want to use algorithms to automatically replenish them. An algorithm is used to predict a stock out for a product in the stores using demand forecasts and actual sales. As soon as the algorithm detects a potential stock out, an order is issued to the distribution centre to replenish the store with the product. So far so good, however unless transportation is nearly free and unlimited (drones maybe?) this way of using algorithms to automate your store replenishment is not that smart. As transportation costs are incurred, the decision to replenish should take into account what other products in the same store should be replenished. Is the total amount of products enough to issue a whole truck? If not, what other stores could the truck attend? In that case the replenishment decision should also incorporate the products to be replenished for the other stores. Next to what to replenish, there is the decision on how much to replenish. This could for example depend on the expected demand for the product in each store, the storage capacity in the store and the revenue it could generate. If a higher margin substitute product is available in the store, you might be even better off by postponing the replenishment of the product as people will buy the more expensive substitute product. As this example shows, in practice a one dimensional trade-off is far too simple. 

The replenishment decision, as many practical decisions, is complex and cannot be made in isolation for a single product. It is depended on other decisions that need to be considered integrally to make the best possible overall decision. #MachineLearning and #AI don’t have the capability to handle this kind of interdependence and complexity in decision making. #Decisionanalytics (aka Prescriptive analytics or Operations Research) however is specifically equipped to model the interdependence between the individual decisions and can leverage the insights from the prediction models to find the guaranteed best possible decision to make.

Want to know more about decision analytics? Check out this video, see my blog or just get in touch.