Knowledge is power
Proudly presenting my article about Big Data, recently published in Computable Magazine (in Dutch):
http://www.computable.nl/artikel/opinie/datamanagement/5244120/4445906/kennis-is-macht.html
Big Data – The answer’s right in front of us
A picture of a race car says not nearly as much as a video. A video consists of dozens of pictures (frames) per second, more material on the same subject. When the image moves, it comes to life and tells a story.
The same evolution is now happening in the field of digital data. The more data on the same subject, the better. With more data, assumptions are more accurate. Predictions come true more often, because such patterns have occurred in the past with the same result.
Over the past fifteen years, the amount of digital data has grown a hundredfold. Each year, this upward trend is becoming steeper and we can speak of an exponential growth. This is due to the arrival of the internet and social media. In addition, large amounts of video and audio are stored digitally. Worldwide, about a hundred hours per minute of footage is uploaded to YouTube. This kind of data is often not analyzed, while valuable information can be extracted doing so.
Big Data is a term that has become very popular in the past years. By immersing in data which we have not previously looked at from a business point-of-view, we can learn much more than before through research and analysis. If the right questions are asked.
Complementing the current data warehousing, Big Data can be of great value in terms of (predictive) analytics. Fueled by huge amounts of data, analysis and discovery of relationships and algorithms are taken to a new level. After an investment for storage and management, the benefits of the process will definitely pay off. Business processes are optimized, increasing turnover is realized and even human lives are saved.
A Big Data research answers more questions than we can ask. It is even possible to start with a Big Data-research without having a specific question in mind. The goal should be: come to new insights. With these insights, specific questions are asked and the intended goals are achieved. First the examination, then the question. The opposite of the usual. The new way.
An intuitive approach to this data is a more efficient way of growing and developing. The great successes achieved with Big Data so far have started with targeted questioning, the old way. Signals leading to a question, a target, which leads to a search. With pin-pointed analysis of digital data on a large scale, there is much more to relate and predict than before. These quests in large volumes of data turn into a journey of discovery that leads to new insights, a new world.
Relationships and connections
Our brain loves correlations. We make connections based on logic and personal experiences, draw conclusions and believe in them. We then adjust our behavior, consciously or unconsciously.
Example:
You visit a restaurant for the first time, then you feel ill the next day. The first thing you blame is the food from yesterday. It's unlikely that you will visit this restaurant again. Our brain establishes a relationship based on logic and believes in it. Statistically, you being ill is much more likely to be caused by shaking hands with someone.
Theorem:
Orange cars need much less maintenance than cars with other colors.
Assumptions:
- "Orange cars are more visible in the evening";
- "Orange is a special color, often chosen by petrol heads, who maintain the car better"
These recognizable assumptions are plausible, however, in reality it can’t be proven that one thing leads to another. With Big Data it’s possible to separate the facts from the rumors. New information can present itself from angles where we have not been able to look at before. The technology is there, and now so is the data. Big Data applications can take us to the truth, one step at a time.
Nowadays we live by facts and statistics. By analyzing click and browsing behavior on websites, it is possible to detect and to act in a timely manner. For example, fraud and other threats. But also early detection of disease. Banks know by the expense patterns when someone gets into financial trouble, before that person knows themselves.
In Great Britain you can get a discount on your car insurance by installing a ‘black box’ in your car. This black box stores data about the car, so accidents and other claims can be better assessed. This will help the insurance company realize cost savings. The advantage of it is passed on to the policyholders in the form of discounts.
Examining existing (own) data
By searching for answers in the data that companies already possess, many questions can be answered that have not been asked before. This works differently for every organization. In many cases this comes from data that is normally not analyzed, while it just may be that the answers to important business questions can be found right there.
This method is most interesting for all companies, regardless of size. When the first step is taken, the results will quickly lead to more research. Even with relatively small investments, it is possible to bring a Big Data project to a successful conclusion.
A telephone company in the Netherlands discovered that the signal strength of their towers is highly dependent on weather conditions. By comparing various signal strength data with weather information they now have a lot of data at their disposal that is used to predict the weather. By predicting relevant aspects of the weather themselves, they now have the ability to act on the effects of weather on the signal strength. Now they have achieved a better signal coverage and customer satisfaction.
Acquiring or producing new data
When the necessary data is not available to answer the question, one can acquire the missing data by looking into other existing sources, such as social media. When this is not possible, the data itself can be produced. By placing additional sensors in equipment for example.
Producing new data is done only for specific purposes (so far). This happens if the data (still) does not exist and therefore can’t be acquired anywhere. Such a process involves major investments and have been conducted by multinational corporations and institutions in the past. Also, scientific projects such as space exploration, are driven by Big Data applications.
A jet-manufacturer has placed additional sensors in the parts they supply to monitor vibration and heat. By storing this data and analyzing it, they have recognized patterns. Now it can be predicted when parts need to be replaced before they become defective during a flight. Maintenances can now be scheduled accordingly to prevent accidents or high repair costs.
Prematurely born babies are very susceptible to infections and can become in a life-threatening situation more quickly if action is not taken on time. By placing sensors on these babies 'vital signs' are registered and patterns are detected. Signals before an infection can now be detected earlier. This way, many lives are saved with the outcome of Big Data analysis.
"Knowledge is power" - Francis Bacon (1561 - 1626)
In a time where knowledge and information was spread personally, not everyone was aware of everything, as is currently the case. We often do not realize that there was a time that people traveled days to pass a message. At that time, information was priceless and it took effort to gather and spread knowledge. Spreading news worldwide was impossible once.
Since the introduction of radio and television, our minds are flooded with information. Today, on the click of a button the whole world knows what someone thinks or feels. Yet relatively little is done with it. This is due to the fact that we now ignore more information than we remember. We choose to do this unconsciously, by the sheer amount of information coming at us daily. The real power of Big Data lies precisely in the analysis of such information. In data we are overwhelmed by and do nothing with.
By using the potential of Big Data, we can collect information that is important to us. The power of Big Data can be excellently used by analyzing, for example, social media data. With Big Data it is possible to recognize patterns in both structured and unstructured data, and to take action.
A pharmaceutical company in the USA has looked at what people express on social media during work. What came forward, were statements like:
"I feel sick"
"I have a runny nose"
This company responded to this in the areas where this happens most, by making more publicity about nasal spray. At the same time they have been watching stocks and ensured that products were not out of stock. In this way, a significantly higher turnover was realized.
By analyzing social media messages political parties can find out which people they can approach and which people they better not. By obtaining this information they choose not to approach certain people, so they don’t activate votes for the opposition.
Social media data is not available to everyone right away, but it can be purchased. Several other open datasets are available on the website of Open Data Netherlands. Big Data is more interesting and more accessible than it seems, looking at the opportunities and successes that have been achieved with it. With a relatively small investment, it is possible to connect external data and convert it into valuable information.
My data, your data
"Privacy is long gone, since GPS in phones."
"You can define and protect your own privacy."
"The collection and use of sensitive data offers many advantages for companies but no benefits for the owner of the data, the consumer."
"Complete privacy is almost impossible, data is collected everywhere."
Aren’t we responsible for publishing our own pictures and video’s online? This does not necessarily mean that companies are allowed to use this information for marketing purposes among others. Or have we already consented to this by accepting the terms and conditions without reading it through?
Online privacy is a sensitive issue and opinions are divided on this subject.
With WiFi-tracking it’s possible to map walking routes in shopping malls. As long as no personal information is linked to the MAC address of our phones, it seems less privacy sensitive than we actually think. And if that happens? How harmful is this to the consumer? As long as people don’t feel bothered, no one really seems to have something against it. This conclusion is based on different viewpoints and opinions on various forums.
In real life, we deal with personal approaches as a result of Big Data applications on a daily basis. Several online retailers and multimedia content providers map our taste and interests and make targeted individual offers about products and services. We are accustomed to this for years now, and often find it enjoyable as well. When such approaches are considered to be undesired or unpleasant, we click it away and we forget it immediately.
The influence and application of Big Data is actually just a big part of our daily lives as the Internet. Therefore, as a modern and intelligent organization, it’s only logical to include Big Data applications as part of our business. Anyone can achieve progress with Big Data, even on a small scale.
The larger the Big Data research, the greater the result.
If the right questions are asked.
Bayram Oztas | Consultant
Data & Analytics
Avanade Netherlands BV
Sources:
- RTL Masterclass: Big Data
- The Big Data Revolution - Kenneth Cukier & Viktor Mayer-Schonberger
