Are you #forecasting the future, the past, or your imagination?

This is an abridged pictureless version of the post on my blog.

Forecasting the future.

The ultimate goal for every CEO, policy maker and marketer.

No wonder that the Big Data Industry is growing by orders of magnitude. The truly impressive industry growth, implied by the numbers from Forbes for 2014 and 2015, comes fairly close to one order of magnitude (and yes, $16B to $125B did catch my eye). Kind of like the growth in Mary Meeker's slides on internet trends. So, before you decide to put a billion euros into the model for future sales, votes, likes, or a human brain for that matter,  what are the basics that you need to be aware of?

Well, first of all, any model is a simplification of yesterday's data. A crude model, such as a simple linear regression, will do a poor job of modelling the data (ask Anscombe). An Artificial Neural Network (or a linear regression with many, many, many interactions) will do a better job, primarily because the model allows for nuances in the data to be reflected by the numerous parameters in the model. But it is still calibrated to fit yesterday's data.

Unless yesterday's data can be used to forecast tomorrow's action, your model will be forecasting the past. With the increasing complexity in business brought on by the rapid adoption of advanced information technology, the likelihood that it does increases by the day. Especially macro predictions are void within months of arrival (as many prominent central bankers and commentators have discovered).

But why do we continue to believe that forecasts matter, when they rarely do? Since the days of Edward Bernays, human expectation has been to known to be a function of available information. Humans may be normally distributed with respect to height, but as regards human expectation, no such luck. That property of our mental models, coupled with rationalisation, group think - all examples of behaviours that are triggered in uncertain environments - and manipulated news feeds, may in fact be a recipe for complete make believe forecasts of what the future holds.

Unless you understand your data, of course.