The Future of Machine Learning and Big Data analysis

The title speaks much for itself. We are entering an era in computational technology, that is making what seemed like the wildest fantasies 10-15 years ago, actually becoming reality. More and more of the tedious and hard manual labor work of checking, analyzing and confirming numbers in computations and systems is being taken over by computers and machines, that can do this at an efficiency surpassing that of humans by tenfolds. But what does this mean for the future of analysis and work done by machines?

The human touch

No matter how quick, precise and superior computers and machines become over humans, there are still certain tasks that humans will be able to do better, due to our ability to improvise and be creative. For now, those are attributes that a computer or machine cannot learn to the same extent as a human. Algorithms and artificial intelligence can create certain structures, rules and assimilations to a lot of data, and find new patterns and ways of doing much of the same that the data shows, but this is reliant on actually being able to obtain and use this data. It cannot just imagine something out of the blue like a human can. Therefore, there will still be required some human interaction, in terms of certain decision making, obtaining certain data for the computer or machine to use, etc.

The obvious relation

But with that in mind, a lot can be said about the future of computational technology. We are approaching a cross road, where humanity must make a decision about what to do in the future with these machines (like the highly debated decision about autonomous vehicle control), as they take over more and more of our work, and seemingly become smarter than us humans. By having access to amounts of data that would baffle any human, a computer can now see, and to some extent understand relations and data patterns that no other human before has been able to see. And by seeing these information relationships, the computers and machines can make new rules and algorithms that help human’s in our everyday lives. This last aspect is what is known as Machine Learning. It is not dependent on having Big Data, but it sure helps the created rules and algorithms to become much better and more accurate in their task, depending on how much data is available. To make a small example, imagine 3 boxes of canned food. 2 are colored green, 1 is colored blue. 1 of the green cans has meat, the other has vegetables. The blue contains a fruit. If one more green can shows up, what does it contain? If the computer had more information about green cans, it could have made an educated guess. And if a blue can shows up, does it contain a fruit? If there had been 10 blue cans, all containing a fruit, then chances are that the newly appeared blue can also contained a fruit.

The greater picture

This was only a small example, and of course a relation that a human could spot in a blink of an eye. But now imagine that there exists millions, if not billions of cans, in so many different colors, that painting each color in a thin line after each other would make a “rainbow” many kilometers long. Try to make an analysis of that, and create similar rules as mentioned before. A human would not be able to figure out how many different colors there were, before a computer would have completed the whole analysis. And the computers analysis would take into account height, circumference, shape, opening mechanism, prices, etc. of the can, and be able to make such a precise rule set, that it would be able to fairly accurately predict a producer, product type, and sale price from the given data of one can compared to the dataset available.

The goal

Now, imagine making such a ruleset for other aspects in human society. When is the best time to travel in traffic, how much electricity is needed at a given time, how often and when are busses or trains most needed, when is a credit card transaction probably a fraud transaction, and so on. Any aspect that requires a known factor, that can be determined from a lot of different data analyzed. However, it does not end there. This aspect is only the “known” relations that a human would ask for. But if Big Data is available about the cans, an analysis could lead to a rule set that could determine whether or not the can is on sale, in a given month of the year, depending on its contents. The original thought of acquiring the dataset about the cans probably didn’t have such an analysis in mind, but with Big Data such an analysis is possible, and highly likely some sort of correlation will appear. Big Data is basically finding these unknown relations that no human would think of finding without the Big Data available.