Which version of ‘Group Think’ will win?

‘Big Data’ is my favourite example of one size fits all appeal and group think.

I can see a lot of confusion emerging within industry between the top down use of technology (big data) promoted by the management fraternity and the bottom up use of technology (the internet of everything) promoted by the technical fraternity, and few voices questioning the applicability of either.

I get the idea of being able to store, process and organise vast amounts of data, no problem. It is the idea of regression analysis of unstructured data on a large scale applied to industries that are based on engineering principles (structured data) that amuses me.

What do we expect to learn about things we understand from first principles? Unless of course you don’t understand first principles.

Clearly I know we can do things more efficiently, that has been the basis of my career. But I don't need regression to tell me how to improve efficiency.

Let me use a sports analogy. I played Rugby for a lot of my life and remain interested in sports science. First principles are as follows: each time the team goes off track go back to basics and correct the inefficiencies that have crept in. Science keeps moving on and we improve efficiency: improved training, injury prevention, improved recovery, and we measure and model performance to learn where we can improve, but we use the analytics to enhance the knowledge of experts.

We don’t gather up all the data on sports in the world and regress it to inform people without the scientific knowledge on how to develop sports teams. Before anyone refers to the movie Moneyball, the movie was about using measured data, not data from regression. It was used to inform management, I agree, but he had specialist knowledge because he had played the game.

However, we could gather up all the data on sports spectators and use it to determine how to promote tickets to the events.

One is first principles, science, based and the other is based on emotional responses. I get the use of regression driven by large scale thought experiments applied to human behaviour, which is of course unstructured but in large numbers exhibits trends (think Isaac Asimov and The Foundation Trilogy).

Big data is of course an aggregation of lots of small data, and back at the sharp end in industry where the technical professionals dwell we have all sorts of 'apps' that structure and analyse small data in close to real time.

If we were only to aggregate the findings from the small data (isn't this called the Internet of Everything or IOE?) we would have our answers without needing to resort to large scale thought experiments on the part of ‘data scientists’ with no industry expertise.

I believe the confusion arises in industry because in between the sharp end and the board we have way too much human behaviour that clouds the messages from the small data, it is called management.

If we more closely connect leadership with the messages from the small data we will eliminate the need for regression of big data in the process industries. This requires management to stop filtering the data, but the huge irony is that management tends to buy into the big data concept of large scale data regression, go figure!

Don’t get me wrong, technical professionals do the same, why simplify when you can complicate?

Surely the Internet of Everything is the answer when you need improvement in performance? Never mind that nobody has properly set up the basics. Anybody can do that, everyone knows that, someone will have done it.

So which version of group think will win?

When influential companies make a buck by promoting group think and you have a market that wants to be seen to be doing the latest thing (tactic) you may end up using the tactics in the wrong place, and worst of all you just added unnecessary complexity, you didn’t even use the new tactic to replace the last tactic.

You must have a strategy to keep it fit for purpose before you embark on technology journeys. The problem is most people don’t seem to understand the difference between strategy and tactics.

If you want to avoid group think all together, be sure you have independent, industry specialist, technology literate thinkers involved in the strategy design. Now where would you get some of those……..?