Reza Zadeh

🧠💬 Memory in LLMs A practical guide showing how to implement conversational memory in LLMs using LangGraph, demonstrated through a therapy chatbot. Features code examples for basic retention, trimming, and summarization approaches. Learn to build memory-aware apps 👉 https://lnkd.in/gybcrV5v

967

21 Comments

It’s time to be real about what HTAP actually means and how to implement it in a way that supports agents and other similar workloads…

11

YellowDog.ai just set a 10x benchmark uplift in scale computing, delivering 40,000 tasks per second (TPS) and managing 100,000 compute nodes in the cloud. What's even more interesting, that's 2x IBM Symphony and opens an intriguing pathway for these until-know closed/captive systems.

11

A new RL algorithm that fixes a hidden flaw in PPO The authors propose CE-GPPO (“Coordinating Entropy via Gradient-Preserving Policy Optimization”), a variant of PPO that restores gradient contributions from clipped actions in the policy update. They argue that traditional clipping discards useful gradient signals from low-probability tokens, which play an important role in controlling the agent’s entropy during training. By bounding those gradients in a controlled way, CE-GPPO maintains exploration–exploitation balance more stably than prior methods. They provide a theoretical analysis showing that CE-GPPO mitigates entropy instability, and empirically test it on mathematical reasoning benchmarks. Their results indicate consistent improvement over strong baselines across different model scales, demonstrating that preserving clipped gradients can lead to better performance in reinforcement learning for reasoning tasks. https://lnkd.in/gYZsJ-8f

1

Interested in our paper about APEX for ultra-large library search, but find it's sitting in one of a dozen open tabs in your browser, waiting for you to get around to reading it? Take a look at our blog post to get a taste of the method, written by paper coauthor Jordi Silvestre-Ryan: https://lnkd.in/g87dPcG4 https://lnkd.in/gGAH4gME

4

#TuesdayPaperThoughts Edition 60: The Art of Scaling RL Compute for LLMs This week's #TuesdayPaperThoughts explores "The Art of Scaling Reinforcement Learning Compute for LLMs" from researchers at Meta, The University of Texas at Austin, University of California, Berkeley, and Harvard University. While pre-training has had its moments with predictable scaling laws, RL training has remained more art than science until now. Key Takeaways: 1️⃣ Predictive Scaling Framework: The work introduces sigmoidal compute-performance curves for RL training that separate asymptotic performance (A) from compute efficiency (B). This framework enables extrapolation from smaller-scale runs to predict performance at larger compute budgets. The team validated this with a massive 100,000 GPU-hour run where predictions from just the first 50k hours closely matched final performance. 2️⃣ Not All Recipes Scale Equally: Methods that look promising at small compute budgets can hit lower performance ceilings at scale. The study reveals that design choices like loss type and FP32 precision shift asymptotic performance, while factors such as loss aggregation and normalization primarily modulate compute efficiency. This explains why some widely-adopted methods plateau unexpectedly. 3️⃣ ScaleRL Recipe: Through systematic ablations consuming 400,000+ GPU-hours, the team developed ScaleRL—combining PipelineRL with 8-step off-policyness, truncated importance sampling (CISPO), FP32 logits computation, and adaptive prompt filtering. ScaleRL achieves A=0.61 asymptotic performance, outperforming DeepSeek's GRPO, Qwen's DAPO, and other prevalent methods on both ceiling and efficiency. The timing couldn't be better. With RL compute budgets exploding (10× increase between model generations for o1→o3 and Grok-3→Grok-4), the field desperately needed this systematic approach. Research Credits: Devvrit K., Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit Dhillon, David Brandfonbrener, Rishabh Agrawal Paper Link: In comments

21

1 Comment

Document-centric tasks sit at the core of many enterprise, business, and government workflows. Search is important, but the real challenge is going beyond retrieval, enabling systems to reason across documents, verify facts, and handle multi-step information tasks. Great to see a new model from Databricks moving in this direction. I expect we’ll see many more models, including open-weight ones, designed specifically for document-driven workflows, an area with huge potential across enterprise and government use cases.

23

1 Comment