Machine Learning and Big Data
Those of us who work in technology find ourselves at a point where we are dealing with two different but connected, disruptive technologies almost every day. "Big Data" and machine learning (along with its big brother, artificial intelligence) are beginning to become a reality in most enterprises.
Big Data is the term used to describe the analysis and correlation of massive sets of data collected from our business operations. These data once threatened to overwhelm us with disparate pieces of information with no clear connections. However, distributed (cloud) computing and the recent advances in machine learning are poised to allow business to make connections and provide insights that were not visible until now.
Machine learning is a form of artificial intelligence whereby a computer algorithm or process can be designed and "trained" to recognize patterns in data and modify their own behavior on the fly based on inferences from those patterns. The most ubiquitous example of this technology in the real-world are the suggestion of search terms as you perform Google searches. More examples of the application of machine learning are found in self-driving cars, high frequency trading on Wall Street, loan approval, fraud detection, diagnosis of medical issues and increasingly in the human-dominated field of customer service.
Chat bots or "conversational agents" that can mimic a natural conversation with another person either through a keyboard interface or using our voices. These technologies have matured to where it's difficult to discern if we are chatting with a human or a machine. Although they are mostly still far from passing a Turing Test (the test judges a machine’s ability to demonstrate intelligent behavior that’s indistinguishable from speaking to a human), the success of these conversation interfaces is not necessarily measured by the ability to fool people but in the delivery of the information or completion of a task we have requested. No one would mistake Alexa or Siri for a human but the success of the Amazon Echo and Google Home and their associated automation ecosystems indicate their time has come.
The application of these technologies is relatively new but businesses are already realizing huge increases in efficiency and seeing real results. There is a rush to develop and implement systems across the enterprise. Automated kiosks are increasingly replacing cashiers at stores, fast food restaurants and even information worker jobs that were previously filled by skilled employees. We can envision a future where many tasks and decision making processes are turned over to the machines with humans being consulted only when there is fallout from a task or specialized input is needed. The requirement for human interaction will diminish as algorithms are perfected.
Currently, we need human programmers and administrators to write code for the systems that increasingly automate our lives. Even the most powerful artificial intelligence systems are still based on algorithms designed by humans, software written by humans and datasets curated and customized by humans. However, as AI eventually reaches the point where it is able to program itself, where will this leave us, the humans and originators of this technology?
