How to break a model in 20 days. A tutorial on production model analytics

It is a story of how we trained a model, simulated production use, and analyzed its gradual decay.

Into the wild: a model in production

Just in 3 short weeks, our model went from this:

To this: 

Why did the quality drop, and how to address it?

In the complete tutorial, I explain how to visualize model performance using Evidently reports and interpret the changes to debug model decay. 

Check it out here: https://evidentlyai.com/blog/tutorial-1-model-analytics-in-production