A Neophyte’s View of Data Analytics

I recently completed a graduate certificate program in Business Analytics from the University of Arkansas. When I first enrolled in this program, I had the naïve impression that I was going to learn how to take a beautiful set of data, run it through some cool algorithms, and produce astounding, business-transforming insights. I was excited about the “Eureka!” moments I was going to have.

But my “Eureka!” moment was realizing that there are no “Eureka!” moments.

When I was an undergraduate (some time ago), I wanted to be an Archaeologist. I had similar unrealistic assumptions about how romantic the life of an Archaeologist could be -- discovering lost cities, solving age-old mysteries, and finding museum-quality artifacts like unblemished pottery and golden statues. Silly me. In reality, a great archaeological discovery can be a potsherd carefully sifted from a midden pile. A trained archaeologist can take that broken piece of pottery and, after careful and thorough analysis, draw educated conclusions -- about the shard, about the item it came from, and about the culture that created and used it. Perhaps this is the calling of the data analyst as well.

Here is what I have learned:

1. Data is dirty and ugly, full of missing, wrong, misleading, unnecessary, and unusable values and will require considerable effort to be transformed into useful input.
2. Even clean data and the best model can produce underwhelming results.
3. Instead of searching for those elusive golden statues, concentrate instead on the shards of truth that are uncovered.
4. Discoveries don’t slap you in the face but are “teased out” with tedious and unrelenting effort.
5. Model output is the beginning, not the end. That unexciting fragment can yield useful and meaningful conclusions.

This hit home for me when I attended an analytics conference and heard a physician admit that their best model was just “OK” but they were still able to turn the information into a clinical protocol that has greatly decreased the number of surgical site infections, improved patient outcomes, and created significant cost savings for the hospital. His conclusion: even mediocre models can produce meaningful results. Sage advice for us neophyte data miners.