An overview of the algorithms behind Recommender Systems

What is a recommender system?

A recommender system is a technology that helps suggest things that one might like according to past behaviour preferences. For example, if you frequently visit or buy house decor products from Jumia, there is a higher chance that you will be getting a wide variety of house decor products in your timeline, each time you log into Jumia

I remember sometime back on twitter when a certain marketing company was tagged by one of its user who was criticizing the company for advertising products the user deemed as "disgusting" or somewhat "immoral". At that time, I had no prior knowledge on the algorithms behind the systems and applications I was using and so to say, if I was the amplifying type, I'd definitely be like the guy on twitter. That being said, let me dive into the reason for this post.

What are the algorithms behind recommender systems?

Recommender systems use machine learning techniques to analyze data and make predictions about what users might like. They use either supervised or unsupervised machine learning techniques, and most times, they would combine the two techniques to improve on the accuracy and performance of the system to recommend to a user.

The most common algorithms used by the recommender systems are:

1. Content-based filtering - This algorithm recommends items to users based on their preferences for certain attributes or characteristics of the items. In simple terms, it uses the content of the items that the user prefers and recommends similar items to the user. There are various methods that are used to check the similarity of the items. The most common are: Cosine similarity and Euclidean distance.Cosine similarity measures the similarity between two items based on their attributes or features. On the other hand, the Euclidean distance measures the dissimilarity between two items based on their attributes or features. The other methods used to check for the similarity of items in the content-based filtering are; pearson correlation and jaccard similarity.
2. Collaborative-filtering- This algorithm recommends items to users using preferences of similar users. In simple terms, we can say it brings about similarity between users and items using collaboration. The two types of collaborative-filtering are user-based collaborative-filtering and item-based collaborative filtering.

i) User-based collaborative filtering: This algorithm looks for users who have similar preferences or behaviors to the target user and recommends items that those similar users have liked or interacted with in the past. In this case, it is a collaboration of users's preferences.

ii) Item-based collaborative filtering: This algorithm looks for items that are similar to the ones the target user has liked or interacted with in the past, and recommends those similar items. In this case, its a collaboration of items' that users prefer.

The similarity methods used by the collaborative-filtering algorithm are cosine similarity, euclidean distance, jaccard similarity and pearson correlation. Cosine similarity is used to measure the similarity between two users or items based on their past interactions. Euclidean distance is used to check the dissimilarity between two users or items based on past interactions. The jaccard similarity is used to measure the similarity between two sets of items and thus, can measure the similarity between two users based on the items they have interacted with in the past. Pearson correlation is used to measure the linear relationship between two variables and in this case, it can be used to measure the relationship between two users based on the items they have previously interaced with.

iii) Matrix factorization is a type of collaborative-filtering that attempts to decompose a user-item matrix into two lower-dimensional matrices representing user and item features or latent features. The basic idea behind matrix factorization is that it can identify latent factors or features that are not explicitly stated in the data, but may be influencing user preferences. For example, in a movie recommendation system, latent features could represent factors such as the genre, director, or actors, which may not be explicitly stated in the data, but could be influencing user preferences.

3. Hybrid recommender systems: These systems combine multiple algorithms to provide more accurate recommendations. For example, a hybrid system might use both content-based filtering and collaborative filtering to suggest items that are similar to what you have liked in the past but also popular among similar users.

These are the most common algorithms behind most recommender systems. There are other algortihms which have been tweaked to perform in accordance the stakeholder's expectations. From this overview, I can affirm that as a reader, you now have a better understanding as to why the tweep received the "immoral" adverts on his timeline.

Recommender systems are important for both users and the marketing companies. The pros of the recommender systems outweigh the cons. For this reason, I highly recommend startups to integrate their websites with this algorithms to improve customer retention and business success overall.