Challenges in TikTok Recommendation Algorithms

Very few applied CS papers exist but this is probably one of the most consequential papers on IR. An answer to why everyone is hooked to TikTok! Excited to share insights about Monolith - ByteDance's groundbreaking real-time recommendation system! Monolith tackles two major challenges in modern recommendation systems: 1. Sparse Feature Handling - Implements a collisionless embedding table using Cuckoo Hashing - Achieves O(1) time complexity for lookups/deletions - Uses two tables with different hash functions to eliminate collisions - Implements smart memory optimization through: • Frequency-based filtering of rare IDs • Automatic expiration of stale embeddings • Probabilistic filtering for further memory reduction 2. Real-time Learning Architecture - Seamlessly integrates batch and online training - Uses Kafka queues for streaming user actions and features - Implements Flink-based online joiner for real-time feature concatenation - Employs intelligent parameter synchronization: • Minute-level updates for sparse parameters • Less frequent updates for dense parameters • Tracks "touched keys" to optimize network usage • On-the-fly updates without service interruption Production Impact: - Significantly outperforms traditional hash-based systems - Shows 14-18% AUC improvement over batch training - Handles terabytes of model parameters efficiently - Successfully deployed in BytePlus Recommend Key Innovation: The system trades traditional reliability constraints for real-time learning capabilities while maintaining robust fault tolerance through daily snapshots - a radical departure from conventional approaches that prioritize frequent checkpointing. This is a fantastic example of how rethinking fundamental assumptions can lead to breakthrough performance in production systems!

The TikTok recommender system is widely regarded as one of the best in the world at the scale it operates at. It can recommend videos or ads, and even the other big tech companies could not compete. Recommending on a platform like TikTok is tough because the training data is non-stationary as a user's interest can change in a matter of minutes and the number of users, videos, and ads keeps changing. The predictive performance of a recommender system on a social media platform deteriorates in a matter of hours, so it needs to be updated as often as possible. TikTok built a streaming engine to ensure the model is continuously trained in an online manner. The model server generates features for the model to recommend videos, and in return, the user interacts with the recommended items. This feedback loop leads to new training samples that are immediately sent to the training server. The training server holds a copy of the model, and the model parameters are updated in the parameter server. Every minute, the parameter server synchronizes itself with the production model. The recommendation model is several terabytes in size, so it is very slow to synchronize such a big model across the network. That is why the model is only partially updated. The leading cause of non-stationary (concept drift) comes from the sparse variables (users, videos, ads, etc.) that are represented by embedding tables. When a user interacts with a recommended item, only the vectors associated with the user and the item get updated, as well as some of the weights on the network. Therefore, only the updated vectors get synchronized on a minute basis, and the network weights are synchronized on a longer time frame.  Typical recommender systems use fixed embedding tables, and the categories of the sparse variables get assigned to a vector through a hash function. Typically, the hash size is smaller than the number of categories, and multiple categories get assigned to the same vector. For example, multiple users share the same vector. This allows us to deal with the cold start problem for new users, and it puts a constraint on the maximum memory that the whole table will use. But this also tends to reduce the performance of the model because user behaviors get conflated. Instead, TikTok uses dynamic embedding sizes such that new users can be added to their own vector. They use a collisionless hashing function so each user gets its own vector. Low-activity users will not influence the model performance that much, so they dynamically remove those low-occurrence IDs as well as stale IDs. This keeps the embedding table small while preserving the quality of the model. Here is the TikTok paper: https://lnkd.in/g9fA62GD! #machinelearning #datascience #artificialintelligence -- 👉 Learn more Machine Learning on my website: https://www.TheAiEdge.io --

Ever wondered how TikTok 📱 keeps you endlessly scrolling? The secret lies in Monolith, their real-time recommender system. Andrej Karpathy, former head of AI at Tesla, famously called TikTok "digital crack" due to its uncanny ability to serve up realtime hyper-personalized content 💯. Why is Monolith so effective? It's built to handle the unique challenges of real-time recommendations: Sparsity: User-item interactions are spread thin across a vast sea of content. 🎥 Dynamism: New users and videos appear constantly, meaning the system must adapt rapidly 🚀. Concept Drift: User preferences evolve, so the system must keep up with changing trends 📈. To address these issues, Monolith employs a collision-less hash table for storing user and item embeddings. This allows efficient updates without sacrificing accuracy ✅. Additionally, frequency filtering and expiration filtering ensure the system remains responsive even as the data grows exponentially 📊. For data scientists and AI engineers, Monolith provides a fascinating case study in real-time recommendation systems. It's a testament to the power of clever architecture, engineering and algorithmic design in creating a product that can truly capture and hold user attention 💡. #datarchitecture #ai #tiktok #datascience