Self-Updating Feedback Loops in LLM Protocols

Last week, I described four design patterns for AI agentic workflows that I believe will drive significant progress: Reflection, Tool use, Planning and Multi-agent collaboration. Instead of having an LLM generate its final output directly, an agentic workflow prompts the LLM multiple times, giving it opportunities to build step by step to higher-quality output. Here, I'd like to discuss Reflection. It's relatively quick to implement, and I've seen it lead to surprising performance gains. You may have had the experience of prompting ChatGPT/Claude/Gemini, receiving unsatisfactory output, delivering critical feedback to help the LLM improve its response, and then getting a better response. What if you automate the step of delivering critical feedback, so the model automatically criticizes its own output and improves its response? This is the crux of Reflection. Take the task of asking an LLM to write code. We can prompt it to generate the desired code directly to carry out some task X. Then, we can prompt it to reflect on its own output, perhaps as follows: Here’s code intended for task X: [previously generated code] Check the code carefully for correctness, style, and efficiency, and give constructive criticism for how to improve it. Sometimes this causes the LLM to spot problems and come up with constructive suggestions. Next, we can prompt the LLM with context including (i) the previously generated code and (ii) the constructive feedback, and ask it to use the feedback to rewrite the code. This can lead to a better response. Repeating the criticism/rewrite process might yield further improvements. This self-reflection process allows the LLM to spot gaps and improve its output on a variety of tasks including producing code, writing text, and answering questions. And we can go beyond self-reflection by giving the LLM tools that help evaluate its output; for example, running its code through a few unit tests to check whether it generates correct results on test cases or searching the web to double-check text output. Then it can reflect on any errors it found and come up with ideas for improvement. Further, we can implement Reflection using a multi-agent framework. I've found it convenient to create two agents, one prompted to generate good outputs and the other prompted to give constructive criticism of the first agent's output. The resulting discussion between the two agents leads to improved responses. Reflection is a relatively basic type of agentic workflow, but I've been delighted by how much it improved my applications’ results. If you’re interested in learning more about reflection, I recommend: - Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023) - Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023) - CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024) [Original text: https://lnkd.in/g4bTuWtU ]

Your LLM just learned to update itself, here's why that changes everything! Forget fine-tuning pipelines and manual data curation. Imagine if your model could write its own training data and decide how to improve itself. Meet SEAL (Self-Adapting LLMs) from the Massachusetts Institute of Technology which Prof Pulkit Agrawal and team presented at NeurIPS 2025. Here's what's happening under the hood:- - The Breakthrough:- Your model encounters a new task → generates a 'self-edit' (instructions to itself) → restructures the data → specifies its own hyperparameters → updates its own weights → becomes permanently better at that task. - No external adaptation modules. No separate fine-tuning infrastructure. The model IS the adaptation engine. Why builders should care right now:- - Reduce your ML Ops complexity:- One model that adapts beats maintaining multiple fine-tuned versions. - Faster iteration cycles:- Model improvement happens at inference time, not after weeks of retraining. - Lower deployment costs:- Self-adaptation means fewer specialized models in production. - Task-specific learning:- The model learns exactly what it needs, when it needs it. The Real Game-Changer:- This isn't just about making models smarter, it's about creating self-improving systems. You're essentially shipping infrastructure that gets better with use. Consider your current ML stack:- How many fine-tuned variants are you maintaining? How much time goes into creating training datasets? SEAL suggests a future where models handle that themselves. The Builder's Dilemma:- With great autonomy comes great uncertainty. When your model controls its own updates:- - How do you validate what it learned? - How do you prevent drift from your original objectives? - How do you audit self-generated training data? This is infrastructure-level innovation that demands infrastructure-level thinking about safety and control. Bottom Line:- We're moving from 'prompt engineering' to 'evolution engineering'. The models that ship in 2026 might look nothing like the models you deployed. Are your systems ready for AI that rewrites itself? If you would like to dive deep, check out:- Paper:- arxiv.org/abs/2506.10943 GitHub:- https://lnkd.in/eqYPHHjU Website:- https://lnkd.in/eKAMbh48 What would you build with a self-adapting model? Drop your wildest use case in the comments! #BuildInPublic #AIEngineering #MLOps #AgenticAI #LLMs #MachineLearning #TechInnovation

Excited to announce my new (free!) white paper: “Self-Improving LLM Architectures with Open Source” – the definitive guide to building AI systems that continuously learn and adapt. If you’re curious how Large Language Models can critique, refine, and upgrade themselves in real-time using fully open source tools, this is the resource you’ve been waiting for. I’ve put together a comprehensive deep dive on: Foundation Models (Llama 3, Mistral, Google Gemma, Falcon, MPT, etc.): How to pick the right LLM as your base and unlock reliable instruction-following and reasoning capabilities. Orchestration & Workflow (LangChain, LangGraph, AutoGen): Turn your model into a self-improving machine with step-by-step self-critiques and automated revisions. Knowledge Storage (ChromaDB, Qdrant, Weaviate, Neo4j): Seamlessly integrate vector and graph databases to store semantic memories and advanced knowledge relationships. Self-Critique & Reasoning (Chain-of-Thought, Reflexion, Constitutional AI): Empower LLMs to identify errors, refine outputs, and tackle complex reasoning by exploring multiple solution paths. Evaluation & Feedback (LangSmith Evals, RAGAS, W&B): Monitor and measure performance continuously to guide the next cycle of improvements. ML Algorithms & Fine-Tuning (PPO, DPO, LoRA, QLoRA): Transform feedback into targeted model updates for faster, more efficient improvements—without catastrophic forgetting. Bias Amplification: Discover open source strategies for preventing unwanted biases from creeping in as your model continues to adapt. In this white paper, you’ll learn how to: Architect a complete self-improvement workflow, from data ingestion to iterative fine-tuning. Deploy at scale with optimized serving (vLLM, Triton, TGI) to handle real-world production needs. Maintain alignment with human values and ensure continuous oversight to avoid rogue outputs. Ready to build the next generation of AI? Download the white paper for free and see how these open source frameworks come together to power unstoppable, ever-learning LLMs. Drop a comment below or send me a DM for the link! Let’s shape the future of AI—together. #AI #LLM #OpenSource #SelfImproving #MachineLearning #LangChain #Orchestration #VectorDatabases #GraphDatabases #SelfCritique #BiasMitigation #Innovation #aiagents

🤔 What if, instead of using prompts, you could fine-tune LLMs to incorporate self-feedback and improvement mechanisms more effectively? Self-feedback and improvement have been shown to be highly beneficial for LLMs and agents, allowing them to reflect on their behavior and reasoning and correct their mistakes as more computational resources or interactions become available. The authors mention that frequently used test-time methods like prompt tuning and few-shot learning that are used for self-improvement, often fail to enable models to correct their mistakes in complex reasoning tasks. ⛳ The paper introduces RISE: Recursive Introspection, an approach to improve LLMs by teaching them how to introspect and improve their responses iteratively. ⛳ RISE leverages principles from online imitation learning and reinforcement learning to develop a self-improvement mechanism within LLMs. By treating each prompt as part of a multi-turn Markov decision process (MDP), RISE allows models to learn from their previous attempts and refine their answers over multiple turns, ultimately improving their problem-solving capabilities. ⛳It models the fine-tuning process as a multi-turn Markov decision process, where the initial state is the prompt, and subsequent states involve recursive improvements. ⛳It employs a reward-weighted regression (RWR) objective to learn from both high- and low-quality rollouts, enabling models to improve over turns. The approach uses data generated by the learner itself or more capable models to supervise improvements iteratively. RISE significantly improves the performance of LLMs like LLaMa2, LLaMa3, and Mistral on math reasoning tasks, outperforming single-turn strategies with the same computational resources. Link: https://lnkd.in/e2JDQr8M

Most LLM agents stop learning after fine-tuning. They can replay expert demos but can’t adapt when the world changes. That’s because we train them with imitation learning—they copy human actions without seeing what happens when they fail. It’s reward-free but narrow. The next logical step, reinforcement learning, lets agents explore and learn from rewards, yet in real settings (e.g. websites, APIs, operating systems) reliable rewards rarely exist or appear too late. RL becomes unstable and costly, leaving LLMs stuck between a method that can’t generalize and one that can’t start. Researchers from Meta and Ohio State propose a bridge called Early Experience. Instead of waiting for rewards, agents act, observe what happens, and turn those future states into supervision. It’s still reward-free but grounded in real consequences. They test two ways to use this data: 1. Implicit World Modeling: for every state–action pair, predict the next state. The model learns how the world reacts—what actions lead where, what failures look like. 2. Self-Reflection: sample a few alternative actions, execute them, and ask the model to explain in language why the expert’s move was better. These reflections become new training targets, teaching decision principles that transfer across tasks. Across eight benchmarks, from home simulations and science labs to APIs, travel planning, and web navigation, both methods beat imitation learning. In WebShop, success jumped from 42 % to 60 %; in long-horizon planning, gains reached 15 points. When later fine-tuned with RL, these checkpoints reached higher final performance and needed half (or even one-eighth) of the expert data. The gains held from 3B to 70B-parameter models. To use this yourself:, here is what you need to do: • Log each interaction and store a short summary of the next state—success, error, or side effect. • Run a brief next-state prediction phase before your normal fine-tune so the model learns transitions. • Add reflection data: run two-four alternative actions, collect results, and prompt the model to explain why the expert step was better. Train on those reflections plus the correct action. • Keep compute constant—replace part of imitation learning, not add more. This approach makes agent training cheaper, less dependent on scarce expert data, and more adaptive. As models learn from self-generated experience, the skill barrier for building capable agents drops dramatically. In my opinion, the new challenge is governance and ensuring they don’t learn the wrong lessons. That means filtering unsafe traces, constraining environments to safe actions, and auditing reflections before they become training data. When rewards are scarce and demonstrations costly, let the agent learn from what it already has, its own experience! That shift turns LLMs from static imitators into dynamic learners and moves us closer to systems that truly improve through interaction, safely and at scale.

A quiet convergence is happening in RL for LLMs: self-distillation and reward-based RL are merging into a single framework (image shown is from RLSD paper, one variant to incorporate self-distillation signals). The emerging answer: let reward serve as the broad correctness anchor, and let self-distillation provide dense token-level correction where the teacher signal is actually trustworthy — gated by quality, not applied uniformly. - SDPO (Hübotter et al.) started the thread by using a model's own feedback-conditioned predictions as a dense self-teacher — no external teacher needed, just richer context at training time. - G-OPD (Yang et al.) reinterpreted on-policy distillation as KL-constrained RL with an implicit reward term and a tunable scaling factor. Their key finding: reward extrapolation (scaling > 1) lets students consistently surpass teachers. - OpenClaw-RL (Wang et al.) demonstrated the split in a live agentic setting — evaluative signals from interactions become scalar rewards via a process reward model, while directive signals from hindsight hints become token-level advantages through on-policy distillation. - REOPOLD (Ko et al.) made the reward interpretation explicit: the teacher-student likelihood ratio is a token-level reward. Adding confidence-sensitive clipping and entropy-driven sampling, a 7B student matched a 32B teacher at 3.3x faster inference. - Nemotron-Cascade 2 (Yang et al., NVIDIA) scaled multi-domain on-policy distillation to competition-level performance — a 30B MoE with only 3B active parameters hit gold-medal level on IMO and IOI using domain-specific intermediate teachers throughout training. - RLSD (Yang et al.) stated the principle most cleanly: decouple direction from magnitude. External reward or verifier signal decides the update sign; self-distillation redistributes token-level credit. The result is a higher convergence ceiling and more stable training than either method alone. - SRPO (Li et al.) operationalized the hybrid by routing samples — successes go to GRPO's reward-aligned reinforcement, failures go to SDPO's targeted logit-level correction. Adding entropy-aware dynamic weighting gives fast early gains from distillation with long-term stability from reward optimization. - Aligning from User Interactions (Kleine Buening et al.) extended the idea beyond synthetic feedback — when users provide follow-ups that signal dissatisfaction, the model's own revised behavior under that context becomes the dense self-teacher, making every conversation a training opportunity.

Google DeepMind's Nested Learning paper (Behrouz et al., 2025) offers a compelling framework for why deep networks learn at multiple timescales. I've translated this into a 𝐩𝐫𝐚𝐜𝐭𝐢𝐜𝐚𝐥 𝐨𝐩𝐞𝐧-𝐬𝐨𝐮𝐫𝐜𝐞 𝐢𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 𝐟𝐨𝐫 𝐋𝐋𝐌𝐬—it works with Qwen, Phi, Gemma, LLaMA, Mistral, and any HuggingFace causal language model. Nested Learning LLM introduces a three-tier adaptation hierarchy: 𝐒𝐥𝐨𝐰 𝐰𝐞𝐢𝐠𝐡𝐭𝐬 (𝐥𝐨𝐰𝐞𝐫-𝐥𝐚𝐲𝐞𝐫 𝐋𝐨𝐑𝐀) preserve foundational linguistic knowledge 𝐌𝐞𝐝𝐢𝐮𝐦 𝐰𝐞𝐢𝐠𝐡𝐭𝐬 (𝐮𝐩𝐩𝐞𝐫-𝐥𝐚𝐲𝐞𝐫 𝐋𝐨𝐑𝐀) handle task-specific adaptation 𝐅𝐚𝐬𝐭 𝐰𝐞𝐢𝐠𝐡𝐭𝐬 (𝐂𝐨𝐧𝐭𝐢𝐧𝐮𝐮𝐦𝐌𝐞𝐦𝐨𝐫𝐲) capture context and instance-specific signals within an episode The memory module functions as a differentiable fast-weight store—it receives hidden representations, computes surprise-gated updates across multiple memory banks, and injects context back into the transformer through a compact gating network. 𝐖𝐡𝐚𝐭 𝐲𝐨𝐮 𝐠𝐞𝐭: • Multi-timescale LoRA with configurable learning rates per layer group • Surprise-driven memory that prioritizes novel information • Test-time adaptation capabilities—feed examples without retraining • Memory-aware generation that updates context on the fly • Full training pipeline with GSM8K, TriviaQA, and CommonsenseQA support 𝗧𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁: meta-learning–style behavior without MAML-style inner loops, in a package that trains only ~2.7M parameters (~0.5% of the base Qwen model). Ideal for few-shot adaptation, continual learning, and test-time reasoning on resource-constrained hardware. Github: https://lnkd.in/e5-GuZ2y #LLM #NLP #NestedLearning #ContinuumMemory

Tired of your LLM just repeating the same mistakes when retries fail? Simple retry strategies often just multiply costs without improving reliability when models fail in consistent ways. You've built validation for structured LLM outputs, but when validation fails and you retry the exact same prompt, you're essentially asking the model to guess differently. Without feedback about what went wrong, you're wasting compute and adding latency while hoping for random success. A smarter approach feeds errors back to the model, creating a self-correcting loop. Effective AI Engineering #13: Error Reinsertion for Smarter LLM Retries 👇 The Problem ❌ Many developers implement basic retry mechanisms that blindly repeat the same prompt after a failure: [Code example - see attached image] Why this approach falls short: - Wasteful Compute: Repeatedly sending the same prompt when validation fails just multiplies costs without improving chances of success. - Same Mistakes: LLMs tend to be consistent - if they misunderstand your requirements the first time, they'll likely make the same errors on retry. - Longer Latency: Users wait through multiple failed attempts with no adaptation strategy.Beyond Blind Repetition: Making Your LLM Retries Smarter with Error Feedback. - No Learning Loop: The model never receives feedback about what went wrong, missing the opportunity to improve. The Solution: Error Reinsertion for Adaptive Retries ✅ A better approach is to reinsert error information into subsequent retry attempts, giving the model context to improve its response: [Code example - see attached image] Why this approach works better: - Adaptive Learning: The model receives feedback about specific validation failures, allowing it to correct its mistakes. - Higher Success Rate: By feeding error context back to the model, retry attempts become increasingly likely to succeed. - Resource Efficiency: Instead of hoping for random variation, each retry has a higher probability of success, reducing overall attempt count. - Improved User Experience: Faster resolution of errors means less waiting for valid responses. The Takeaway Stop treating LLM retries as mere repetition and implement error reinsertion to create a feedback loop. By telling the model exactly what went wrong, you create a self-correcting system that improves with each attempt. This approach makes your AI applications more reliable while reducing unnecessary compute and latency.

𝘞𝘦 𝘢𝘴𝘴𝘶𝘮𝘦 𝘈𝘐 𝘮𝘰𝘥𝘦𝘭𝘴 𝘯𝘦𝘦𝘥 𝘮𝘢𝘴𝘴𝘪𝘷𝘦, 𝘩𝘶𝘮𝘢𝘯-𝘤𝘶𝘳𝘢𝘵𝘦𝘥 𝘥𝘢𝘵𝘢𝘴𝘦𝘵𝘴 𝘵𝘰 𝘪𝘮𝘱𝘳𝘰𝘷𝘦. 𝘞𝘩𝘢𝘵 𝘪𝘧 𝘵𝘩𝘦𝘺 𝘤𝘰𝘶𝘭𝘥 𝘨𝘦𝘵 𝘴𝘮𝘢𝘳𝘵𝘦𝘳 𝘦𝘯𝘵𝘪𝘳𝘦𝘭𝘺 𝘰𝘯 𝘵𝘩𝘦𝘪𝘳 𝘰𝘸𝘯? New research from Carnegie Mellon University shows LLMs can bootstrap their own learning, using nothing but a single prompt. This is a huge deal because data acquisition and labeling for post-training are massive bottlenecks, demanding immense engineering effort. A truly self-learning pipeline could fundamentally change the economics of AI development. In their paper, "𝐒𝐞𝐥𝐟-𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐢𝐧𝐠 𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐌𝐨𝐝𝐞𝐥𝐬 (𝐒𝐐𝐋𝐌)," researchers tackle this problem. They designed an asymmetric self-play framework where a 'Proposer' model generates questions and a 'Solver' model answers them. Both are trained via reinforcement learning without any ground-truth data. The ingenious part is the reward mechanism. - For tasks like math, correctness is determined by a majority vote over multiple generated answers. - For coding, the Proposer also generates unit tests, and the Solver is rewarded for passing them. This forces the Proposer to generate an adaptive curriculum of problems that are always at the edge of the Solver's ability. The results are striking: - A 3B parameter model boosted its accuracy on algebra problems by 16 percentage points (from 44% to 60%). - On complex three-digit multiplication, accuracy jumped nearly 16 percentage points (from 79.1% to 94.8%). The takeaway: This moves beyond static synthetic data; it's a dynamic, self-improving loop. This research paves the way for more autonomous AI systems that can master new domains with minimal human intervention, dramatically reducing the reliance on costly training datasets. It's a foundational step toward models that can truly think and learn for themselves. #AI #LLM #ReinforcementLearning #MachineLearning #Research

Most Retrieval-Augmented Generation (RAG) pipelines today stop at a single task — retrieve, generate, and respond. That model works, but it’s 𝗻𝗼𝘁 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁. It doesn’t adapt, retain memory, or coordinate reasoning across multiple tools. That’s where 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝗥𝗔𝗚 changes the game. 𝗔 𝗦𝗺𝗮𝗿𝘁𝗲𝗿 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗳𝗼𝗿 𝗔𝗱𝗮𝗽𝘁𝗶𝘃𝗲 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 In a traditional RAG setup, the LLM acts as a passive generator. In an 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 system, it becomes an 𝗮𝗰𝘁𝗶𝘃𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺-𝘀𝗼𝗹𝘃𝗲𝗿 — supported by a network of specialized components that collaborate like an intelligent team. Here’s how it works: 𝗔𝗴𝗲𝗻𝘁 𝗢𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗼𝗿 — The decision-maker that interprets user intent and routes requests to the right tools or agents. It’s the core logic layer that turns a static flow into an adaptive system. 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗠𝗮𝗻𝗮𝗴𝗲𝗿 — Maintains awareness across turns, retaining relevant context and passing it to the LLM. This eliminates “context resets” and improves answer consistency over time. 𝗠𝗲𝗺𝗼𝗿𝘆 𝗟𝗮𝘆𝗲𝗿 — Divided into Short-Term (session-based) and Long-Term (persistent or vector-based) memory, it allows the system to 𝗹𝗲𝗮𝗿𝗻 𝗳𝗿𝗼𝗺 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲. Every interaction strengthens the model’s knowledge base. 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗟𝗮𝘆𝗲𝗿 — The foundation. It combines similarity search, embeddings, and multi-granular document segmentation (sentence, paragraph, recursive) for precision retrieval. 𝗧𝗼𝗼𝗹 𝗟𝗮𝘆𝗲𝗿 — Includes the Search Tool, Vector Store Tool, and Code Interpreter Tool — each acting as a functional agent that executes specialized tasks and returns structured outputs. 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 𝗟𝗼𝗼𝗽 — Every user response feeds insights back into the vector store, creating a continuous learning and improvement cycle. 𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 Agentic RAG transforms an LLM from a passive responder into a 𝗰𝗼𝗴𝗻𝗶𝘁𝗶𝘃𝗲 𝗲𝗻𝗴𝗶𝗻𝗲 capable of reasoning, memory, and self-optimization. This shift isn’t just technical — it’s strategic It defines how AI systems will evolve inside organizations: from one-off assistants to adaptive agents that understand context, learn continuously, and execute with autonomy.