BIG Data Analysis
What is the point of data? I mean we all know data is a kind of information that we have about a thing. It is generated either quantitatively or qualitatively. It is common sense. When I say what is the point of data, I mean what purpose does it serve? The common answer is to help make better decisions about an activity. Decisions that are based upon evidence, trends and insights backed up by logic and rationale. It is a multidimensional process.
Data has been generated for centuries now. We have used it to make our lives better, our vehicles faster, our houses stronger and what not. Like every other thing that came into existence and by their own capacities are existing even now, data has withstood the test of time. This also proves that the concept of data is reliable as it formed and hardened by the test of time.
Data has gone from being strictly naturalistic to quantitative to qualitative to mixed. The latest incarnation of data is in the form of Big Data. There is a peculiar reason why it is so called as well. Let’s delve in this to understand it a bit better. Let’s start with a basic definition of Big Data. The definition goes as follows
“extremely large data sets that may be analysed computationally to reveal patterns, trends, and associations, especially relating to human behaviour and interactions”
Data sets are different collections of data. Being mindful of that, it is to be seen that they can be analysed by us to reveal trends and insights that relate to human behaviour and interactions. Another factor to be kept in mind is that, yes data has been collected earlier and data has been analysed earlier as well by humans. The difference however is that the data this time is too enormous for humans themselves to analyse and come up with insights.
We have to take the aid of Artificial Intelligence and Supercomputers to come up with insights from millions of data sets. There are various loops in this simple looking model when it comes to practical side of world.
We begin by observing the size of data sets that are available. These data sets can be diverse in nature as well. Therefore, recording this data and comparing them becomes a problem. Further, given the mammoth set of data that is available, which goes into the tera without breaking a sweat also makes storing the data a huge problem. Storage becomes a problem as the cost of storing goes up and we are talking data storage at a level where it becomes quite expensive.
Keeping numerous problem like this, bright minds in various fields of sciences have come up with innovative solutions and are still coming up better solutions every day. The problem is far from solved however, that hasn’t stopped the hard-working people of this field. Big Data has provided us with solutions not only to the problems we knew about but also problems that may have come out in the future, that we may not even have known about. This is what data scientists refer to as ‘insights’ that come out of Big Data processed by Supercomputers.
This is just one of the ways Big Data comes up to the aid of humans. There are various other ways in which Big Data can help us. The most popular example of Big Data interacting with another field of science is IBM’s Watson. Watson is the best example of Artificial Intelligence of machine meeting the high-speed processing of numerous processors churning up meaningful data through Big Data techniques. Watson was a challenge IBM took on to make a quantum leap in the field of AI.
The status of Watson is that it has outsmarted a human at the game of Jeopardy. Big data has a lot to offer and is just brimming with possibilities. We just need to tap into it. It is upon us to make the most the next step into data generation by using Big Data to aid us in endeavours.
Anand Rai
