Opposite Proverbs problem in analytics

Just like the proverbs "pen is mightier than sword" and "action speaks louder than words" we have many similar analogy in world of analytics. One of this contradiction is in:

  • Six Sigma
  • Fisher's Theorem

Now, very simply put the idea behind Six Sigma is to reduce variation; Where as Fisher's theorem states more the variation, higher is the rate of adaptation; i.e. as per Fisher encouraging variation in environment works better.

Wait, whats going on? We have a rub here. So we have this problem of opposite proverbs. We need to remember there is the No Free Lunch Theorem (and it states that for certain types of mathematical problems, the computational cost of finding a solution, averaged over all problems in the class, is the same for any solution method). Thus we should not believe in one algorithm, or one theorem that will work on all problems. So we need to back up and think about "which settings", "on which landscape" these two theorems going to work?

And there you have it, landscapes where you have spent a lot of time to find the peak (like medical procedures, or production line, or reporting & MIS) Six Sigma is the best fit as you would like to hang out on that peak. But if the environment is churning, the world is changing then you would like to have more variation and Fisher's Theorem comes to rescue.

Thus on a dancing landscape, where things are constantly evolving your Marketing Strategies got to change to match customer preferences. The landscape is advancing you can't rely on a checklist. So you have to encourage new ideas so that you can learn and adapt from them. We need to have multiple models and then adjunct models based on their assumptions and settings where they can work.

When you are not at the peak, you want to move towards it. In Six Sigma setting you presume you are.

Thanks,

Abir Mukherjee.

Very well written. On related lines the problem many times in the real world is that you assume you are operating at the peak but you aren’t. Analytically this results in data blind spots which limits your visibility outside your local optimal making it cost prohibitive to operate outside. That’s why good companies always ensure they have randomized test beds. Unfortunately, too many do not and become obsolete over time.

To view or add a comment, sign in

More articles by Abir Mukherjee

  • Riding the Elevator in the House of Generative AI

    On a quiet Sunday morning, picture yourself standing in the lobby of a building that hums faintly, as if wired with…

    9 Comments
  • The Second Climb: When Growth Isn’t a Ladder, But a Reboot

    At some point in our careers, we begin to expect stability — like we’ve earned the right to cruise at altitude. Titles…

    2 Comments
  • When Reports Don’t Think: The Limits of Static Views in a Dynamic Business

    In today's data-rich world, most organizations have successfully deployed self-service BI tools. Dashboards are…

    3 Comments
  • Arch supporter

    Today I woke up to find myself transformed into my own arch supporter. By 7:30 I had selected my beverage for the…

    2 Comments
  • Daylight to me

    When I give you the task I take it you have a clear understanding of the task, and I won't interfere with the way you…

    7 Comments
  • Predictive & Causal Relationships

    With the rise of Big Data predictive modeling has grown explosively in the last decade, while causal models have taken…

    1 Comment
  • Seduction of the Underdog

    Tactics, Innovation & Evolution The phenomenon to switch our allegiances for a singular contest just based on who's up…

    2 Comments
  • p-value = Indian Head Nod ?

    I like to think of p-value in statistics similar to an Indian head nod. I might expect to answer yes or no in relation…

    1 Comment
  • Teach Your MODELs Not To Memorise NOISE.

    Design, train and tune a Machine Learning Model is very much like raising a kid through infancy, childhood and then…

    3 Comments
  • Empiricism vs. Rationalism : What sort of Analytical thinker are you ?

    There are two broad camps of philosophies behind analytics that people fall into. And that defines how do you like to…

    3 Comments

Others also viewed

Explore content categories