Lies, Damned Lies and Data

The saying goes, "There are lies, damned lies, and statistics." That statement by Mark Twain is one of many he made that would have delighted a Cynic philosopher (yes, it is a philosophy and not simply an adjective for anyone found disagreeable). I am sure he must have been thinking of marketers or politicians when he crafted this line. Today, most of us are so jaded by the disingenuousness that comes at us from all directions we may no longer chuckle when we read Twain's statement.  

However, in general, Twain needed a different word than statistics. After attending the first day of statistics in college and getting an overview of the content for the semester, I left in a cold sweat, went to my advisor's office, and changed my major to avoid that class. Statistics is quite impressive in its capabilities. Marketers and politicians seldom dive into the world of statistics. I will reserve judgment on why this is.

Though arguably Twain stands alone at such a level that suggesting a rewrite or edits to any of his words borders on heresy, if he were alive today, I feel certain he would have replaced "statistics" with "data." The word "data" did exist in Twain's world and had since 1624 but did not gain its current level of use until after the dawn of the technology age in 1942. So, Twain did the best he could with what he had, and the point made remains painfully true today.

This suggested substitution of "data" for "statistics" is supported by something called the DIKW pyramid or hierarchy, used to show the interrelationship and ascending value of data, information, knowledge, and wisdom. Data inhabits the lowest value rung. Data is available everywhere. Four years ago, our species was creating 2.5 quintillion bytes of data daily (2,500,000,000,000,000,000). Because such a massive amount of datum is – on the whole – not worth much, we have analysis as a methodology for searching the haystack to find the golden needle. An analysis is the first step in the alchemy of converting data into something useful. Thanks to algorithms applied by clever people using massive computing capability, huge data stores can be analyzed. 

From this process, we glean information though the value and usefulness of it are still unknown. Typically, information provides generalizations, the meaning of which is unknown. Big data companies come in all shapes and sizes, but if all they offer is analysis, what you get for what you pay is unlikely to meet your expectations. Enter knowledge. Knowledge is created when subject matter expertise is applied to the analysis. Big data and its offspring, information, are relatively worthless as tools without knowledge.

I recently conducted a research project for one of my clients in partnership with a leading survey company. The research resulted in 28 data tables from which we derived 200 distinct information points after analysis. However, the information points only took on value when the subject matter expertise of our team in digital delivery channels was applied. A generalist big data company with practices stretching across multiple industries may ask its clients to provide the subject matter expertise. Think of them as Swiss Army knives, and their clients are the one that decides which tool and how to apply it. If you want some help with the heavy lifting that requires decades of experience in a niche or vertical. Thinking of engaging a big data company? Always make sure your big data analysis comes with a heaping side of subject matter expertise. Don't be afraid to interview them to see if they even know half of what you do about your industry or, better, can tell you something you did not know. If not, keep looking.

Wisdom is more esoteric than data, information, and expertise. Wisdom refers to the practical application of a person's knowledge in those circumstances where good may result. Wisdom is different from knowledge; in the same way, learning about the law is different from arguing a court case. If you haven't been in a situation to test your knowledge, you cannot provide any wisdom as to how what you are proposing will play out. Wisdom is the final mile in the value chain that starts with those 2.5 quintillion bytes of data humans create daily. 

Wisdom, a la Mark Twain, is rare indeed. It should be the first consideration in selecting the leadership for any endeavor. If you are keeping score, then you decide how often that is the case.