How to Improve Agent Performance With Llms

Last week, I described four design patterns for AI agentic workflows that I believe will drive significant progress: Reflection, Tool use, Planning and Multi-agent collaboration. Instead of having an LLM generate its final output directly, an agentic workflow prompts the LLM multiple times, giving it opportunities to build step by step to higher-quality output. Here, I'd like to discuss Reflection. It's relatively quick to implement, and I've seen it lead to surprising performance gains. You may have had the experience of prompting ChatGPT/Claude/Gemini, receiving unsatisfactory output, delivering critical feedback to help the LLM improve its response, and then getting a better response. What if you automate the step of delivering critical feedback, so the model automatically criticizes its own output and improves its response? This is the crux of Reflection. Take the task of asking an LLM to write code. We can prompt it to generate the desired code directly to carry out some task X. Then, we can prompt it to reflect on its own output, perhaps as follows: Here’s code intended for task X: [previously generated code] Check the code carefully for correctness, style, and efficiency, and give constructive criticism for how to improve it. Sometimes this causes the LLM to spot problems and come up with constructive suggestions. Next, we can prompt the LLM with context including (i) the previously generated code and (ii) the constructive feedback, and ask it to use the feedback to rewrite the code. This can lead to a better response. Repeating the criticism/rewrite process might yield further improvements. This self-reflection process allows the LLM to spot gaps and improve its output on a variety of tasks including producing code, writing text, and answering questions. And we can go beyond self-reflection by giving the LLM tools that help evaluate its output; for example, running its code through a few unit tests to check whether it generates correct results on test cases or searching the web to double-check text output. Then it can reflect on any errors it found and come up with ideas for improvement. Further, we can implement Reflection using a multi-agent framework. I've found it convenient to create two agents, one prompted to generate good outputs and the other prompted to give constructive criticism of the first agent's output. The resulting discussion between the two agents leads to improved responses. Reflection is a relatively basic type of agentic workflow, but I've been delighted by how much it improved my applications’ results. If you’re interested in learning more about reflection, I recommend: - Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023) - Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023) - CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024) [Original text: https://lnkd.in/g4bTuWtU ]

Agentic AI Design Patterns are emerging as the backbone of real-world, production-grade AI systems, and this is gold from Andrew Ng Most current LLM applications are linear: prompt → output. But real-world autonomy demands more. It requires agents that can reflect, adapt, plan, and collaborate, over extended tasks and in dynamic environments. That’s where the RTPM framework comes in. It's a design blueprint for building scalable agentic systems: ➡️ Reflection ➡️ Tool-Use ➡️ Planning ➡️ Multi-Agent Collaboration Let’s unpack each one from a systems engineering perspective: 🔁 1. Reflection This is the agent’s ability to perform self-evaluation after each action. It's not just post-hoc logging—it's part of the control loop. Agents ask: → Was the subtask successful? → Did the tool/API return the expected structure or value? → Is the plan still valid given current memory state? Techniques include: → Internal scoring functions → Critic models trained on trajectory outcomes → Reasoning chains that validate step outputs Without reflection, agents remain brittle, but with it, they become self-correcting systems. 🛠 2. Tool-Use LLMs alone can’t interface with the world. Tool-use enables agents to execute code, perform retrieval, query databases, call APIs, and trigger external workflows. Tool-use design involves: → Function calling or JSON schema execution (OpenAI, Fireworks AI, LangChain, etc.) → Grounding outputs into structured results (e.g., SQL, Python, REST) → Chaining results into subsequent reasoning steps This is how you move from "text generators" to capability-driven agents. 📊 3. Planning Planning is the core of long-horizon task execution. Agents must: → Decompose high-level goals into atomic steps → Sequence tasks based on constraints and dependencies → Update plans reactively when intermediate states deviate Design patterns here include: → Chain-of-thought with memory rehydration → Execution DAGs or LangGraph flows → Priority queues and re-entrant agents Planning separates short-term LLM chains from persistent agentic workflows. 🤖 4. Multi-Agent Collaboration As task complexity grows, specialization becomes essential. Multi-agent systems allow modularity, separation of concerns, and distributed execution. This involves: → Specialized agents: planner, retriever, executor, validator → Communication protocols: Model Context Protocol (MCP), A2A messaging → Shared context: via centralized memory, vector DBs, or message buses This mirrors multi-threaded systems in software—except now the "threads" are intelligent and autonomous. Agentic Design ≠ monolithic LLM chains. It’s about constructing layered systems with runtime feedback, external execution, memory-aware planning, and collaborative autonomy. Here is a deep-dive blog is you would like to learn more: https://lnkd.in/dKhi_n7M

We’re entering an era where AI isn’t just answering questions — it’s starting to take action. From booking meetings to writing reports to managing systems, AI agents are slowly becoming the digital coworkers of tomorrow!!!! But building an AI agent that’s actually helpful — and scalable — is a whole different challenge. That’s why I created this 10-step roadmap for building scalable AI agents (2025 Edition) — to break it down clearly and practically. Here’s what it covers and why it matters: - Start with the right model Don’t just pick the most powerful LLM. Choose one that fits your use case — stable responses, good reasoning, and support for tools and APIs. - Teach the agent how to think Should it act quickly or pause and plan? Should it break tasks into steps? These choices define how reliable your agent will be. - Write clear instructions Just like onboarding a new hire, agents need structured guidance. Define the format, tone, when to use tools, and what to do if something fails. - Give it memory AI models forget — fast. Add memory so your agent remembers what happened in past conversations, knows user preferences, and keeps improving. - Connect it to real tools Want your agent to actually do something? Plug it into tools like CRMs, databases, or email. Otherwise, it’s just chat. - Assign one clear job Vague tasks like “be helpful” lead to messy results. Clear tasks like “summarize user feedback and suggest improvements” lead to real impact. - Use agent teams Sometimes, one agent isn’t enough. Use multiple agents with different roles — one gathers info, another interprets it, another delivers output. - Monitor and improve Watch how your agent performs, gather feedback, and tweak as needed. This is how you go from a working demo to something production-ready. - Test and version everything Just like software, agents evolve. Track what works, test different versions, and always have a backup plan. - Deploy and scale smartly From APIs to autoscaling — once your agent works, make sure it can scale without breaking. Why this matters: The AI agent space is moving fast. Companies are using them to improve support, sales, internal workflows, and much more. If you work in tech, data, product, or operations — learning how to build and use agents is quickly becoming a must-have skill. This roadmap is a great place to start or to benchmark your current approach. What step are you on right now?

𝐈 𝐡𝐚𝐯𝐞 𝐬𝐩𝐞𝐧𝐭 𝐭𝐡𝐞 𝐥𝐚𝐬𝐭 𝐲𝐞𝐚𝐫 𝐡𝐞𝐥𝐩𝐢𝐧𝐠 𝐄𝐧𝐭𝐞𝐫𝐩𝐫𝐢𝐬𝐞𝐬 𝐦𝐨𝐯𝐞 𝐟𝐫𝐨𝐦 "𝐈𝐌𝐏𝐑𝐄𝐒𝐒𝐈𝐕𝐄 𝐃𝐄𝐌𝐎𝐒" 𝐭𝐨 "𝐑𝐄𝐋𝐈𝐀𝐁𝐋𝐄 𝐀𝐈 𝐀𝐆𝐄𝐍𝐓𝐒".  The pattern is always the same:  Teams nail the LLM integration and think the hard part is done, then realize they have built 20% of what production actually requires. 𝐇𝐞𝐫𝐞 𝐢𝐬 𝐰𝐡𝐲 𝐞𝐚𝐜𝐡 𝐛𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐛𝐥𝐨𝐜𝐤 𝐦𝐚𝐭𝐭𝐞𝐫𝐬: Reasoning Engine (LLM): Just the Beginning • Interprets intent and generates responses • Without surrounding infrastructure, it is just expensive autocomplete • Real engineering starts when you ask: "How does this agent make decisions it can defend?" Context Assembly: Your Competitive Moat • Where RAG, memory stores, and knowledge retrieval converge • Identical LLMs produce vastly different results based purely on context quality • Prompt engineering does not matter if you are feeding the model irrelevant information Planning Layer: What to Do Next • Breaks goals into steps and decides actions before acting • Separates thinking from doing • Poor planning = agents that thrash or make circular progress Guardrails & Policy Engine: Non-Negotiable • Defines what APIs the agent can call, what data it can access • Determines which decisions require human approval • One misconfigured tool call can cascade into serious business impact Memory Store: Enables Continuity • Short-term state + long-term memory across interactions • Without it, every conversation starts from zero • Context window isn't memory it's just scratchpad Validation & Feedback Loop: How Agents Improve • Logging isn't learning • Capture user corrections, edge cases, quality signals • Best teams treat every interaction as potential training data Observability: Makes the Invisible Visible • When your agent fails, can you trace exactly why? • Which context was retrieved? What reasoning path? What was the token cost? • If you can not answer in under 60 seconds, debugging will kill velocity Cost & Performance Controls: POC vs Product • Intelligent model routing, caching, token optimization are not premature they are survival • Monthly bills can drop 70% with zero accuracy loss through smarter routing What most teams miss: They build top-down (UI → LLM → tools)  when they should build bottom-up (infrastructure → observability → guardrails → reasoning). These 11 building blocks are not theoretical. They are what every production agent eventually requires either through intentional design or painful iteration. 𝐖𝐡𝐢𝐜𝐡 𝐛𝐥𝐨𝐜𝐤 𝐚𝐫𝐞 𝐲𝐨𝐮 𝐜𝐮𝐫𝐫𝐞𝐧𝐭𝐥𝐲 𝐮𝐧𝐝𝐞𝐫𝐢𝐧𝐯𝐞𝐬𝐭𝐢𝐧𝐠 𝐢𝐧? ♻️ Repost this to help your network get started ➕ Follow Anurag(Anu) Karuparti for more PS: If you found this valuable, join my weekly newsletter where I document the real-world journey of AI transformation. ✉️ Free subscription: https://lnkd.in/exc4upeq #GenAI #AIAgents

Large language models (LLMs) can improve their performance not just by retraining but by continuously evolving their understanding through context, as shown by the Agentic Context Engineering (ACE) framework. Consider a procurement team using an AI assistant to manage supplier evaluations. Instead of repeatedly inputting the same guidelines or losing specific insights, ACE helps the AI remember and refine past supplier performance metrics, negotiation strategies, and risk factors over time. This evolving “context playbook” allows the AI to provide more accurate supplier recommendations, anticipate potential disruptions, and adapt procurement strategies dynamically. In supply chain planning, ACE enables the AI to accumulate domain-specific rules about inventory policies, lead times, and demand patterns, improving forecast accuracy and decision-making as new data and insights become available. This approach results in up to 17% higher accuracy in agent tasks and reduces adaptation costs and time by more than 80%. It also supports self-improvement through feedback like execution outcomes or supply chain KPIs, without requiring labeled data. By modularizing the process—generating suggestions, reflecting on results, and curating updates—ACE builds robust, scalable AI tools that continuously learn and adapt to complex business environments. #AI #SupplyChain #Procurement #LLM #ContextEngineering #BusinessIntelligence

Most LLM agents stop learning after fine-tuning. They can replay expert demos but can’t adapt when the world changes. That’s because we train them with imitation learning—they copy human actions without seeing what happens when they fail. It’s reward-free but narrow. The next logical step, reinforcement learning, lets agents explore and learn from rewards, yet in real settings (e.g. websites, APIs, operating systems) reliable rewards rarely exist or appear too late. RL becomes unstable and costly, leaving LLMs stuck between a method that can’t generalize and one that can’t start. Researchers from Meta and Ohio State propose a bridge called Early Experience. Instead of waiting for rewards, agents act, observe what happens, and turn those future states into supervision. It’s still reward-free but grounded in real consequences. They test two ways to use this data: 1. Implicit World Modeling: for every state–action pair, predict the next state. The model learns how the world reacts—what actions lead where, what failures look like. 2. Self-Reflection: sample a few alternative actions, execute them, and ask the model to explain in language why the expert’s move was better. These reflections become new training targets, teaching decision principles that transfer across tasks. Across eight benchmarks, from home simulations and science labs to APIs, travel planning, and web navigation, both methods beat imitation learning. In WebShop, success jumped from 42 % to 60 %; in long-horizon planning, gains reached 15 points. When later fine-tuned with RL, these checkpoints reached higher final performance and needed half (or even one-eighth) of the expert data. The gains held from 3B to 70B-parameter models. To use this yourself:, here is what you need to do: • Log each interaction and store a short summary of the next state—success, error, or side effect. • Run a brief next-state prediction phase before your normal fine-tune so the model learns transitions. • Add reflection data: run two-four alternative actions, collect results, and prompt the model to explain why the expert step was better. Train on those reflections plus the correct action. • Keep compute constant—replace part of imitation learning, not add more. This approach makes agent training cheaper, less dependent on scarce expert data, and more adaptive. As models learn from self-generated experience, the skill barrier for building capable agents drops dramatically. In my opinion, the new challenge is governance and ensuring they don’t learn the wrong lessons. That means filtering unsafe traces, constraining environments to safe actions, and auditing reflections before they become training data. When rewards are scarce and demonstrations costly, let the agent learn from what it already has, its own experience! That shift turns LLMs from static imitators into dynamic learners and moves us closer to systems that truly improve through interaction, safely and at scale.

Over the past few weeks, I validated several patterns that reveal how AI agents truly behave in production. Autonomy is impressive, but structure still delivers the most consistent results. In a traditional LLM workflow where logic and reasoning are fully orchestrated, the same model ran twice as fast and used twelve times fewer tokens than an agentic setup. Efficiency scales best when reasoning is guided, not left open-ended. When deterministic logic was moved into the orchestration layer, the agent gained flexibility, but it came at a cost: more time and higher token usage. Predictable performance, yet less efficient overall. The biggest insight came from reasoning models themselves. GPT 5, with its superior compression and contextual efficiency, outperformed GPT 4o not because it was larger, but because it reasoned more precisely. What my findings validated: For simple and well-defined use cases, LLM workflows can achieve over 99% reliability without complex agent logic. A verifier layer - a lightweight “check my work” agent, can further improve reliability and confidence. For complex, critical, or regulated processes, orchestration remains faster, cheaper, and more auditable. Autonomy sounds exciting, but it isn’t always the optimal path. The smartest systems know when to act independently and when to rely on structured reasoning. AI agents perform best within boundaries that balance adaptability with control. Use them where discovery and contextual reasoning create value. Rely on orchestration where precision, governance, and cost efficiency are non-negotiable.

This is my first time building a services company. In the past, my co-founder and I only built product companies. Product companies are infinitely scalable theoretically and VC fund-able, while services companies are not. And being honest, it's more fun to work on a product than a service. However, with AI, this has completely changed. There is a huge demand for services at the moment, so the real challenge is, how do you build an effective IT services company in the age of AI? Delivering services is messy and a lot of unscalable hard work. Think about it like this: You need to bring founder level energy to every project to be effective and deep dive into the details for every new client (and do this over and over and over again). So how do you solve this? The answer is, leveraging AI and LLMs to streamline the service delivery process. What do I mean by this? Implementing LLMs and Agents internally to help go from AI automation idea to SOW all the way to technical tickets being scoped out for engineers to work on. Here is what our current requirements gathering process looks like, using LLMs and AI Agents (this would’ve taken weeks and hours of meetings): 1. Clients interact with our conversational agent to explain their problem, workflow, and goals (Think of this as your technical project manager deep dive) 2. The agent translates this into a draft proposal with potential AI automation plans & options 3. Our team reviews, adjusts for budget + timeline, and turns it into a polished SOW 4. Client reviews the plans → approves the plans → we move to finalized SOW 5. From there, we auto-generate tickets in our project management system so engineers and PMs can execute immediately Clients go from idea to signed proposal + scoped plan in days. We still keep humans in the loop for quality and nuance. To me, this is a small but important breakthrough. One of the best applications of AI / LLMs is improving how IT projects & services are delivered, where the AI does the heavy lifting around requirements gathering, scoping and finalized planning. What scared me away from services before (i.e. the upfront grind) is now the part I actually look forward to. I can see this pattern applying to any services-like business, not just IT services delivery (think Legal / Accounting etc.).

Most AI agents fail not because of the model. But because you're drowning your AI Agent in garbage context. The fix isn't a better LLM. It's better 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠. Here's the framework you can use to fix your agents: • If context is too large → COMPRESS Stop sending 200k tokens when 5k will do. Summarize, rank by relevance, and trim aggressively. Compression isn't just cheaper, it's clearer. • If context is irrelevant → SELECT Use RAG with smart filters. Keep only what matters. Teams see 3× higher accuracy just by pruning noise. Less is always more. • If context is conflicting → ISOLATE Split your workflows. One agent, one clear task, one clean workspace. Stop making a single brain juggle contradictions. • If context needs memory → WRITE Save stable facts in a persistent layer. Build an AGENTS md file with reusable rules. The smartest context is the one that remembers. The pattern is simple: Your agent fails → Label the failure → Apply the fix → Watch accuracy jump. Next time your agent fails, don’t swap the model. Label the failure: Large / Irrelevant / Conflicting / Need Memory. Apply the fix. Then watch your accuracy and cost graphs flip.