Different Types of Machine Learning: Part 3 – Supervised Learning: Decision Trees, Random Forests, SVM, k-Nearest Neighbors, and GBM

1. In the previous article, we discussed Linear Regression, Polynomial Regression, and Logistic Regression.
2. Today, we continue with a brief review of different supervised learning techniques.

Decision Trees

1. The simplest version of decision trees is something people use regularly in their daily lives.
2. For example, deciding whether to purchase a specific car: If the car is broken — No. Otherwise, if the year is < 2020 — No. Otherwise...
3. A Decision Tree is a tree-like model that makes decisions based on feature values. It splits data into branches based on conditions, eventually leading to predictions.
4. Can be used for Classification (customer segmentation, recommendation systems) and Regression (predicting house prices, predicting employee salaries).

Random Forests

1. Decision Trees are a great tool, but their usage is limited.
2. Smart researchers found that, instead of using just one tree, it makes sense to use multiple decision trees and combine them to improve prediction accuracy.
3. And so, the Random Forests method was created.
4. In some cases, Random Forests can work better and faster than neural networks.
5. This approach can be used in finance, healthcare, and bioinformatics.

Support Vector Machines (SVM)

1. Primarily used for classification, but can also handle regression tasks.
2. SVM acts like a smart "boundary drawer."
3. It finds the best line (or boundary) to separate data points into different groups (classes).
4. For example, if you have data on cats and dogs, SVM will try to draw a line that separates all the cats on one side and all the dogs on the other.
5. Example use cases include text classification and image recognition.

k-Nearest Neighbors (k-NN)

1. In k-NN, each data point is classified based on its nearest "neighbors" in the data.
2. Imagine you have a new fruit, and you want to classify it as an apple or an orange based on its size and color. You measure the size and color of the nearest fruits (neighbors) around it. If most of them are apples, you classify the new fruit as an apple. If most are oranges, you label it an orange.
3. k-NN is useful when you have well-separated data.
4. Can be used for pattern recognition, recommendation systems.

Gradient Boosting Machines (GBM)

1. Imagine you’re learning a new skill, like cooking. The first time you try a recipe, it’s not perfect, but you learn from your mistakes. The next time, you fix some of those mistakes, getting a bit closer to a perfect dish. Over time, with each attempt, you keep getting better by learning from what went wrong before.
2. GBM works similarly by "boosting" its accuracy in stages, learning from its errors at each step.
3. GBM builds a series of simple models, often decision trees. Each weak learner on its own isn’t very accurate, but together, they build a stronger, more accurate model.
4. Each new model focuses on correcting the mistakes made by the previous model.
5. Can be used in predictive analytics, financial modeling.

Neural Networks

1. Some neural networks are also part of supervised learning. However, we don’t plan to discuss them in this article. Neural networks will be covered in a separate article.