Big Data, Big Deal

According to IBM, we’re creating 2.5 quintillion bytes of data each and every day—“so much that 90% of the data in the world today has been created in the last two years alone.” The ever- increasing volume and detail of data from global financial systems, web-based retail transactions, social media chatter, the growing expanse of digital everything, and a myriad of other sources have propelled us into the era of “big data.” It’s the latest and greatest technology trend and an exciting one at that.

In an O’Reilly Radar post called “An Introduction to the Big Data Landscape,” technologist and writer Edd Dumbill defines the catch-all term “big data” as “data that exceeds the processing capacity of conventional database systems.” Big data is data that originates from sources like personal and corporate email, automobile sensors, XML feeds, Apache weblogs, online purchases, and the growing expanse of digital everything. Its sheer volume, velocity, and variety make big data different from anything most organizations have ever had to deal with from an information management perspective. As Dumbill accurately points out, “big data” is either too big, moves too fast, or doesn’t always look like the kind of data we’re used to putting into our corporate databases, so we have to choose alternative ways to process and store it if we stand any chance of finding value in it.

For forward-looking business leaders, the value of big data lies in the promise of new discovery and insight that can fuel greater operational performance—not from traditional business intelligence, but from an emerging and related discipline Dumbill calls “data science”: the merging of “math, programming, and scientific instinct.” Big data evangelists relish the prospect of answering questions that, in the past, were simply beyond reach.

In fact, The New York Times cited a recent World Economic Forum report entitled “Big Data, Big Impact” that declares data “a new class of economic asset, like currency or gold.” Gartner says that “‘Big data’ initiatives create significant opportunities for business and IT leaders….” And Forrester analyst Vanessa Alvarez sums it all up by simply stating, “Big data means big value.”

So, to a lot of folks, big data is a really big deal.

Here’s the thing.

In a study conducted by Accenture, 53 percent of the middle managers say that less than half of the information they receive on a routine basis today [emphasis mine] through traditional BI channels is even valuable, and 42 percent say that at least once a week they inadvertently use the wrong information to guide decisions. For far too many executives, right information is either incomplete or missing entirely from the decision-making process. Unlike those evangelists who see new and endless possibilities in big data, most of the executives I work with relish the prospect of using the data they already have to answer basic questions around operational performance to drive some value.

In my view, big data should only be a big deal for organizations already experienced in using information to better manage and optimize the enterprise. For them, big data provides a big opportunity for discovery and insight that can drive incredible innovation. Everyone else should keep big data on their radar but focus first on building a BI competency that is hyper-responsive, hyper-agile, and hyper-flexible.

Excerpted from the new book, Hyper: Changing the way you think about, plan, and execute business intelligence for real results, real fast! by Greg Steffine (Foreword by Boris Evelson, Vice President and Principal Analyst, Forrester Research).

Available on Amazon in Kindle, paperback, and hardcover editions.

Other articles I've written:

• The Problem With BI
• Great Facilitators Drive BI Success
• A Consultant Isn't Your Competitor

I am a business intelligence strategist, solution delivery leader, and Amazon bestselling author. For more than 25 years now, I've had the privilege of working with some of the most recognizable names in business—from mid-market growth organizations to the Fortune 500. My passion is to help business leaders use information to create and maximize value. Visit steffine.com to learn more. Follow me @GregSteffine.