THE BIG DATA

Some quick facts about Big Data…

• More than 2.5 billion gigabytes of data is generated every single day.
• It is estimated that 90% of all of the world data today was produced within the last two years. This is the exponential growth of data!
• Most of the data stored is in digital format. It accounted to about 94% by 2007.
• Facebook alone collects 500 terabytes of data daily, including 2.5 billion pieces of content, 2.7 billion “likes” and 300 million photos. Facebook has admitted to have stored 100 petabytes of photo and video as of 2012.
• Google processes about 3.5 billion requests per day.

40 zettabytes of data will be in existence by 2020. Three years ago, the entire World Wide Web is estimated to contain approximately 500 exabytes – which is 5 billion gigabytes, but only half of one zettabyte! 40 zettabytes is, therefore, 400 billion gigabytes! This is the “Big Data.”

The term Big Data has been a buzz word since quite a long time. Yet, it doesn’t have a standard definition that is accepted universally. But it is necessary to have a fair idea of what big data is before going any further. The description given by Wikipedia is a good point to start with.  "Any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications."  There are three prominent aspects that make the regular data the “Big Data”.  They are Volume, Velocity and Variety. One need not dig deep to understand this aspects of the data. Just recollect how data was few years ago. Compare it to what it is now. The numerous ways in which data is being generated and consumed every second! The varieties and the pace at which it is being generated! This has been made possible by the ever growing “affordable” technology. Apart from the technology itself, the term affordable is very important here. For data to exist, it has to be created, accessed and stored.

A 5 MB hard disk drive from Apple cost $3,500 in 1981. That’s $700,000 per GB. Considering that we now have tiny, cheap USB sticks that can hold up to 64 GB of data, which is about 1,600 times more than a normal hard disk drive in 1990 (40 MB), and 12,800 times more than the first consumer hard disk drive in 1980, things have certainly moved forward. Accessing and Storing data has never been easier and it is only going to get much easier. Do you know how much data you access and create every day?

So where and how the big data is being used?

This is the impending question you all are going to have now. Big data is critical in almost every industry, be it healthcare, politics, sports, retailing, movies, governments, telecommunication, insurance, security etc. We are in an era where quality information is regarded as the biggest wealth. Big data provides this quality information.

Using big data, retailers can predict what products will sell, telecom companies can predict if and when a customer might switch carriers, and car insurance companies understand how well their customers actually drive and can offer insurance policies accordingly.

It’s also used to optimize business processes. Retailers are able to optimize their stock levels based on what’s trending on social media, what people are searching for on the web, or even weather forecasts. Supply chains can be optimized so that delivery drivers use less gas and reach customers faster. The personalized recommendations you get on YouTube, Netflix or any other online shopping website are a result of the big data analytics.

But big data goes way beyond shopping and consumerism. Big data analytics enable us to find new cures and better understand and predict the spread of diseases. Police forces use big data tools to catch criminals and even predict criminal activity. Los Angeles Police Department, New York Police Department and the Chicago Police Department have been doing this successfully. The credit card companies use big data analytics to detect fraudulent transactions.

 A number of cities are even using big data analytics with the aim of turning themselves into Smart Cities, where a bus would know to wait for a delayed train and where traffic signals predict traffic volumes and operate to minimize jams. In Boston, the power of big data has been leveraged to identify the exact location of the pot holes on the road to reduce the road hazards that happen. Big data can also be used to predict the outcomes of elections, games etc. This has been done with great accuracy during the 2012 U.S Presidential elections and the Summer Olympics. All these have been successfully implemented so far and the use of big data is only going to spread to more such fields.

Apart from the vast range of benefits big data offers, it also has its own share of concerns. Security is the biggest of all those. Big data could be your “Big Brother”. Lot of private and sensitive data is being deciphered from the big data. Misusing it can lead to unprecedented threat to the security of customers, companies, governments etc. Bad use of data can be worse than no data at all.

The biggest reason why big data is going to be important is because it has essentially changed the way business is done and the current trend is only going to grow. As the tools required to capture and analyze the big data get more and more cheap and accessible, its use is only going to be increased. Whether you like it or not, you have to embrace the use of big data to sustain and benefit in the long run.

All said, the important thing to realize is that big data is not inherently good or evil. It depends on how one uses it!

“Data Accessed is Data Created.”