Handling the over-complex decisions in data science using Bayesian Optimization

The practical application of Bayesian Optimization has been seen more and more frequently recently. And it's pleasing to see such a mathematically profound algorithm work for real-world problems.

One of the very useful applications so far has been the hyper-parameter tuning for AI-models, which opened the door to fast Auto-ML applications today. With increasing complexity of learning algorithms, the tuning tasks had become over-complex, where you needed to optimize your overall system over hundreds/thousands of parameters at once. Yet that's not been the only challenge. Another fact which makes the whole problem harder is that you collect your data incrementally, in other words you actually work in an experimental setup. At this point, Bayesian optimization helps us to optimize our cost-function in an incremental manner, in a most cost-efficient way.

A few good examples of this solution in actual work are Google's Machine Learning Engine or Microsoft's Automated ML Capability Azure. Both of these solutions are exciting examples of using machine learning to solve complexities caused by machine learning! Sounds like a little AI :)

But tuning the hyper-parameters of ML pipelines is just one right use-case where you can exploit Bayesian optimization. Actually, any complex parametric system which needs to be optimized incrementally in an experimental setup would be a potential candidate to apply Bayesian Optimization on.

Another good example came at this point from Facebook, which they published lately on their research blog: Efficient tuning of online systems. In that novel approach, they apply Bayesian Optimization in an A/B testing setup to improve their products as well optimize their highly parametric back-end systems with as few shots as possible (remember, cost-efficiency!). This can be a guiding work for optimizing any complex digital marketing system, where you work in A/B testing setup.

It will surely be interesting to see how Bayesian optimization will be applied for similar problems in the near-future and how widely it will be in use.