Humanizing “Machine Learning”

Would you ask your child to look through millions of pictures of cats to understand what a cat looks like? Moreover, would you isolate them from the rest of the society during this process? Of course not!

Why should we treat machine learning models any differently?

Machine Learning(ML) algorithms show human like behavior already, performing better when their peer models are diverse, learning from each other, and showing different perspectives based on their education(training data)!

In the past, scientists have implemented concepts like 'wisdom of the crowds’ and ‘diversity’ into their models, and have also been successful in teaching them to focus on the mistakes they made.

We will explore these ideas together, and I will then show you how I believe all of us can contribute to their further development, by reflecting on how we best learn as humans.

CART, one of the oldest algorithms in ML, suffered from the demerit that its predictions were very sensitive to changes in the input data-set. Recall the tale of the frog in the well, who repeatedly makes incorrect judgments about the world, due to its limited experiences.

In order to overcome its inherent flaws, a technique called 'bootstrapping' was developed. This technique involved creating over 1000 similar but slightly different data-sets, in order to train CART models on the modified data-sets. But why train so many models? Consider classroom style education, where all students pools in their ideas to develop a consensus on the answer to a particularly challenging problem. The imperfections in knowledge of one child are overlooked by the "wisdom of the crowd", leading to better predictions.

Learning from our mistakes is something we humans do instinctively. While preparing for the GRE, a triathlon, or a piano recital, we always highlight the parts we are weak at, and emphasize getting those parts right the next time we practice! Each iteration of the learning process involves reflecting on which parts we did good at, and which need improvement.

Emphasising the importance of fixing mistakes is the core idea of the 'ADABoost' and other boosting algorithms. Adding 'weights' to the answers that the ML model got incorrectly ensure that those data points are looked at more carefully in its next round of learning. The fact that the model highlights its own mistakes and learns from them often leads to better model predictions, thus it should not be surprising that the ‘ADABoost’ algorithm is considered as one of the top contenders in supervised learning(a type of ML problem) settings.

This trip to the past was extremely informative, as it helped us ground our understanding of how ML models were improved upon with the help of ideas from pedagogy. Cognizant of this approach, let us look at some of the most cutting edge research and identify how more sophisticated pedagogical ideas are being implemented in the Machine Learning models of today.

A fascinating development using these ideas in recent history is that of Active Learning. In order to understand the brilliance of this idea, let us go back to the first question of this article.

Would you ask your child to look through millions of pictures of cats to understand what a cat looks like?

Imagine that I’ve already trained my machine learning model to understand that the following picture is a cat.

(Image 1: A really pretty cat. Credits: Image by cocoparisienne from Pixabay) 

In your opinion, which of the following two images(if shown next) would the model learn more from?

(Image 2: A similar photo to the original. Credits: Image by cocoparisienne from Pixabay)

(Image 3: A different cat from a different angle. Credits: Image by Sbringser from Pixabay)

Of the two images, the first one is very similar to the original photo, and the model would learn a lot more if it were provided the second image.

This idea of deciding the next image(training instance) based on the learning potential is the core idea behind Active Learning, and it leads to reduced data processing, training costs and time, while maintaining the same level of accuracy!

This glimpse into current state-of-the-art is certainly exciting, but what about the future?

Humans are excellent at transferring skills and knowledge we learn from one domain to another. Can we mimic how humans transfer knowledge across domains, to allow a single model to develop a multi-faceted understanding of the world around it? Is there a way we can allow ML models to talk to and learn from each other, much like we do in real life? Scientists are already working on developing answers to these questions, and we are sure to see an exciting decade of research ahead of us.

Machine Learning models are a blessing to all fields of engineering, and are proving to be an invaluable tool for many fields of research across science and medicine. Due to its inherent mathematical nature, there seems to be a barrier of entry that doesn’t allow us to contribute to its developments easily.

However, I argue that almost all advancements in machine learning have come from simple pedagogical ideas.

A systematic breakdown of best educational practices will be a good starting point to design new algorithms that are faster and more accurate. For example, I best learn by covering my study material in sprints, where I skim the material quickly. Having understood how the material is structured, I revisit the harder parts of the material, deepening my understanding. I then repeat this process over and over until I am satisfied with my level of expertise in that field.

By systematically writing my learning process down, I can now pick at these ideas, to see which ones have already been implemented, and which ones can be translated to better models.

Why don’t you take this opportunity to collaborate with a statistician/ machine learning engineer/ mathetematician and share your vision on how humans should be educated? On how you best learn? Since the ideas you end up discussing will be abstract, they have the potential to be applied across all fields and applications of machine learning!

You never know, your next coffee shop conversation might be the inspiration for the next generation of our fascinating mechanical progeny.

Notes: Thank you for reading to the end, dear reader. If you would like to follow up on this article and discuss Machine Learning or anything interesting, please feel free to reach out to me!

Gratitude to cocoparisienne and Sbringser for providing access to the cat photos. I had a fun time searching for the right cat pictures for this article. Links to their work are down below:

https://pixabay.com/photos/cat-animal-cat-portrait-cat-s-eyes-1508613/ https://pixabay.com/photos/cat-animal-cute-pet-feline-kitty-618470/ https://pixabay.com/photos/cat-animal-cat-portrait-mackerel-1045782/