Do you understand Big Data?

I’ll admit it: Big Data is not my favorite term. It really trivializes and summarizes the trend too far, creating misconceptions of the true nature of this incredible shift in our technology, culture and world.

Despite the fact that I might bill myself as a “big data” expert, I wish we could come up with a better term. If you’re not sure what I’m talking about, you’re not alone. 

1. Some people simply ignore big data. Bad move. Data and the ability to turn data into business value is becoming increasingly important in any sector. In business, information is power, and big data is providing information we couldn’t have dreamed of collecting or analyzing just a few short years ago. Ignoring it is sticking your head in the sand; this is a trend that is not going away.
2. Most people think big data is about data. Interestingly, it is not; it's about what you do with it. I’ve long argued that simply collecting data or even analyzing it isn’t the end game of a big data strategy. Instead, it’s about how you use the information you glean from the data. It’s about the processes you improve, the decision making you enable, the business value you add. Data for data’s sake is meaningless. Without valuable interpretation and smart implementation, big data projects are a costly and ridiculous waste of time.
3. Most people think big data is about quantity. Big data got its name because advances in technology seemed to quite suddenly allow us to collect and analyze much greater quantities of data than ever before. But part of that was also because we gained the ability to analyze new types of data — especially unstructured data. Before, the only usable data was the kind that fit neatly into the rows and columns of a database. Today, we can analyze large blocks of text, like books and journals, video, photos, audio, health records, and more. So big data isn’t just about the volume of data; it is equally about the variety of data to which we now have access.
4. Some people think the more data you get your hands on the better. Some companies have become data hoarders, collecting and storing as much data as they possibly can against the future chance that they might need it. But this quickly becomes a very costly enterprise. Data storage isn’t free, and as the size of your “collection” grows exponentially with every year, so does the cost. In addition, searching and analyzing those vast quantities of data become more challenging and require more resources. Instead of hoarding data, collect only what you really need and what makes business sense. I always advocate understanding the questions you want to answer before designing a big data project so that you can remain lean and focused on the outcomes.
5. Some people think big data means you need to collect and store large amounts of your own data. Actually, no. As companies and organizations began to view their data as the business commodity it is, a market emerged where organizations can buy, sell, and trade data. In addition, a great deal of valuable data is being collected and shared by open government data initiatives, scientific research organizations, and and other not-for-profit agencies. Many organizations will find that the data they need already exists — or much of it already exists, greatly reducing what they specifically need to collect and store.

Hopefully, this will help clear up some of the confusions you might have about big data and its misleading name, the basic understanding of the concept and term will help to navigate the inevitable changes to come.