There is life (or analysis) beyond Statistics and Machine Learning?

When I got to know Statistics and Machine Learning, I was obsessed to apply what I had learned in all analysis I was doing. It consumed me to the point that I used to think that the results would not be good enough if I didn't do so. In my mind that was what separates a good job of those who would be remembered by the Board.

Experience obtained through errors (and also the "beatings" we took doing this) help us to expand our thinking. The techniques linked to Statistics and Machine Learning are very important, but they should be regarded as any tool among the available options.

There's no question the comfort, speed and status that a Ferrari can give us, but it makes sense to spend so much when we only need to go through a block? Walking or cycling could address the need with less cost and more effectively. However, we sometimes choose a Ferrari in analysis.

The ideas behind Data Science encourage us to go beyond the scope of Statistics and Machine Learning. However, I fear that people start to consider this discipline as the "Holy Grail" of analysis (as is sometimes done by Statistics and Machine Learning defenders) and be surprised again. We can see it coming: "my analysis will not be good enough if I do not apply Data Science (however I do not know exactly what that means)".

Building an analysis model does not necessarily require the application of any of these disciplines. There are extremely sophisticated models based on simple arithmetic and logic operations. The secret may lie in knowledge and insights behind these models.

I remember that one of most successful models that I built, conceived to assess losses from operations by fictitious members, can be summarized as the junction of databases and the development of a rule (no statistics, no machine learning, only logic) to mark the improper members and, consequently, the improper operations.

By the way, we do not always need models with high level of complexity to achieve great results. More important than the techniques we use in the analysis is the result and the benefit it will bring.

The Statistics and Machine Learning advocates may argue that all these points are covered by these disciplines. This is true, but they are not the only keepers of solutions; we end up with this perception when we face so many fiery and fanciful speeches.

Statistics and Machine Learning can help us a lot. However, it is our logical thinking ability that combines all the knowledge to find the answers to what we seek. After all, our brain is the main tool that we have.

Portuguese version here.