How to Improve AI Assistant Natural Language Processing

Last week, I described four design patterns for AI agentic workflows that I believe will drive significant progress: Reflection, Tool use, Planning and Multi-agent collaboration. Instead of having an LLM generate its final output directly, an agentic workflow prompts the LLM multiple times, giving it opportunities to build step by step to higher-quality output. Here, I'd like to discuss Reflection. It's relatively quick to implement, and I've seen it lead to surprising performance gains. You may have had the experience of prompting ChatGPT/Claude/Gemini, receiving unsatisfactory output, delivering critical feedback to help the LLM improve its response, and then getting a better response. What if you automate the step of delivering critical feedback, so the model automatically criticizes its own output and improves its response? This is the crux of Reflection. Take the task of asking an LLM to write code. We can prompt it to generate the desired code directly to carry out some task X. Then, we can prompt it to reflect on its own output, perhaps as follows: Here’s code intended for task X: [previously generated code] Check the code carefully for correctness, style, and efficiency, and give constructive criticism for how to improve it. Sometimes this causes the LLM to spot problems and come up with constructive suggestions. Next, we can prompt the LLM with context including (i) the previously generated code and (ii) the constructive feedback, and ask it to use the feedback to rewrite the code. This can lead to a better response. Repeating the criticism/rewrite process might yield further improvements. This self-reflection process allows the LLM to spot gaps and improve its output on a variety of tasks including producing code, writing text, and answering questions. And we can go beyond self-reflection by giving the LLM tools that help evaluate its output; for example, running its code through a few unit tests to check whether it generates correct results on test cases or searching the web to double-check text output. Then it can reflect on any errors it found and come up with ideas for improvement. Further, we can implement Reflection using a multi-agent framework. I've found it convenient to create two agents, one prompted to generate good outputs and the other prompted to give constructive criticism of the first agent's output. The resulting discussion between the two agents leads to improved responses. Reflection is a relatively basic type of agentic workflow, but I've been delighted by how much it improved my applications’ results. If you’re interested in learning more about reflection, I recommend: - Self-Refine: Iterative Refinement with Self-Feedback, by Madaan et al. (2023) - Reflexion: Language Agents with Verbal Reinforcement Learning, by Shinn et al. (2023) - CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing, by Gou et al. (2024) [Original text: https://lnkd.in/g4bTuWtU ]

Are humans 5X better than AI? This paper is blowing up (not in a good way) The recent study claims LLMs are 5x less accurate than humans at summarizing scientific research. That’s a bold claim. But maybe it’s not the model that’s off. Maybe it's the AI strategy, system, prompt, data... What’s your secret sauce for getting the most out of an llm? Scientific summarization is dense, domain-specific, and context-heavy. And evaluating accuracy in this space? That’s not simple either. So just because a general-purpose LLM is struggling with a turing style test... doesn't mean it can't do better. Is it just how they're using it? I think it's short sighted to drop a complex task into an LLM and expect expert results without expert setup. To get better answers, you need a better AI strategy, system, and deployment. Some tips and tricks we find helpful: 1. Start small and be intentional. Don’t just upload a paper and say “summarize this.” Define the structure, tone, and scope you want. Try prompts like: “List three key findings in plain language, and include one real-world implication for each.” The clearer your expectations, the better the output. 2. Test - Build in a feedback loop from the beginning. Ask the model what might be missing from the summary, or how confident it is in the output. Compare responses to expert-written summaries or benchmark examples. If the model can’t handle tasks where the answers are known, it’s not ready for tasks where they’re not. 3. Tweak - Refine everything: prompts, data, logic. Add retrieval grounding so the model pulls from trusted sources instead of guessing. Fine-tune with domain-specific examples to improve accuracy and reduce noise. Experiment with prompt variations and analyze how the answers change. Tuning isn’t just technical. Its iterative alignment between output and expectation. (Spoiler alert: you might be at this stage for a while.) 4. Repeat Every new domain, dataset, or objective requires a fresh approach. LLMs don’t self-correct across contexts, but your workflow can. Build reusable templates. Create consistent evaluation criteria. Track what works, version your changes, and keep refining. Improving LLM performance isn’t one and done. It’s a cycle. Finally: If you treat a language model like a magic button, it's going to kill the rabbit in the hat. If you treat it like a system you deploy, test, tweak, and evolve It can retrieve magic bunnies flying everywhere Q: How are you using LLMs to improve workflows? Have you tried domain-specific data? Would love to hear your approaches in the comments. 

Prompt optimization is becoming foundational for anyone building reliable AI agents Hardcoding prompts and hoping for the best doesn’t scale. To get consistent outputs from LLMs, prompts need to be tested, evaluated, and improved—just like any other component of your system This visual breakdown covers four practical techniques to help you do just that: 🔹 Few Shot Prompting Labeled examples embedded directly in the prompt help models generalize—especially for edge cases. It's a fast way to guide outputs without fine-tuning 🔹 Meta Prompting Prompt the model to improve or rewrite prompts. This self-reflective approach often leads to more robust instructions, especially in chained or agent-based setups 🔹 Gradient Prompt Optimization Embed prompt variants, calculate loss against expected responses, and backpropagate to refine the prompt. A data-driven way to optimize performance at scale 🔹 Prompt Optimization Libraries Tools like DSPy, AutoPrompt, PEFT, and PromptWizard automate parts of the loop—from bootstrapping to eval-based refinement Prompts should evolve alongside your agents. These techniques help you build feedback loops that scale, adapt, and close the gap between intention and output

LLM literacy is now part of modern UX practice. It is not about turning researchers into engineers. It is about getting cleaner insights, predictable workflows, and safer use of AI in everyday work. A large language model is a Transformer based language system with billions of parameters. Most production models are decoder only, which means they read tokens and generate tokens as text in and text out. The model lifecycle follows three stages. Pretraining learns broad language regularities. Finetuning adapts the model to specific tasks. Preference tuning shapes behavior toward what reviewers and policies consider desirable. Prompting is a control surface. Context length sets how much material the model can consider at once. Temperature and sampling set how deterministic or exploratory generation will be. Fixed seeds and low temperature produce stable, reproducible drafts. Higher temperature encourages variation for exploration and ideation. Reasoning aids can raise reliability when tasks are complex. Chain of Thought asks for intermediate steps. Tree of Thoughts explores alternatives. Self consistency aggregates multiple reasoning paths to select a stronger answer. Adaptation options map to real constraints. Supervised finetuning aligns behavior with high quality input and output pairs. Instruction tuning is the same process with instruction style data. Parameter efficient finetuning adds small trainable components such as LoRA, prefix tuning, or adapter layers so you do not update all weights. Quantization and QLoRA reduce memory and allow training on modest hardware. Preference tuning provides practical levers for quality and safety. A reward model can score several candidates so Best of N keeps the highest scoring answer. Reinforcement learning from human feedback with PPO updates the generator while staying close to the base model. Direct Preference Optimization is a supervised alternative that simplifies the pipeline. Efficiency techniques protect budgets and service levels. Mixture of Experts activates only a subset of experts per input at inference which is fast to run although the routing is hard to train well. Distillation trains a smaller model to match the probability outputs of a larger one so most quality is retained. Quantization stores weights in fewer bits to cut memory and latency. Understanding these mechanics pays off. You get reproducible outputs with fixed parameters, bias-aware judging by checking position and verbosity, grounded claims through retrieval when accuracy matters, and cost control by matching model size, context window, and adaptation to the job. For UX, this literacy delivers defensible insights, reliable operations, stronger privacy governance, and smarter trade offs across quality, speed, and cost.

For years now, prompt engineering shaped how people worked with large language models. It was about finding the right phrasing to get predictable outputs. That approach worked for small tasks, but as models turned into agents that plan, use tools, and retain memory, the limits became obvious. One of Anthropic’s latest articles “𝘌𝘧𝘧𝘦𝘤𝘵𝘪𝘷𝘦 𝘤𝘰𝘯𝘵𝘦𝘹𝘵 𝘦𝘯𝘨𝘪𝘯𝘦𝘦𝘳𝘪𝘯𝘨 𝘧𝘰𝘳 𝘈𝘐 𝘢𝘨𝘦𝘯𝘵𝘴”, introduces the next phase in this evolution, called context engineering. It explains that success now depends on how well we manage what goes inside the model’s attention window rather than how we word instructions. Anthropic describes context as everything the model sees while reasoning, including prompts, data, retrieved results, tool outputs, and message history. Every token consumes a portion of the model’s attention, and as the window expands, its focus gradually weakens. The new challenge is to curate that space carefully. Below are the main lessons from Anthropic’s work that stand out for anyone building practical AI systems. 1. Treat context as a limited resource. Adding more information does not improve accuracy. Use only what directly supports the current reasoning step. 2. Write system prompts like structured briefs. Divide them into clear parts for background, instructions, tools, and expected output. 3. Build small, distinct tools. Each tool should solve one problem and return compact, unambiguous results. 4. Use a few canonical examples instead of long lists of edge cases. Examples should teach reasoning, not overwhelm the model with detail. 5. Retrieve data just in time rather than all at once. Lightweight references such as file paths or queries keep the model’s focus clear. 6. Compact long interactions. Summarize the conversation and restart with the essentials so that the model stays coherent over long sessions. 7. Store information outside the context window. Structured notes or state files help maintain continuity across projects. 8. Use sub-agents for large tasks. Specialized agents can work on details while a coordinator manages direction and synthesis. 9. Balance autonomy with reliability. Some data should stay fixed for consistency, while other parts can be fetched dynamically when needed. 10. Focus attention on signal, not volume. Every token should contribute to the next action or decision. Prompt writing will still matter, but the real skill now lies in shaping context and deciding what enters the model, what stays out, and how information evolves as the agent works. The next generation of LLM Agents will depend less on clever wording and more on precise design of memory, retrieval, and context. Context engineering is becoming the foundation for reliable agents that think and act across long horizons with consistency and purpose.

You’re doing it. I’m doing it. Your friends are doing it. Even the leaders who deny it are doing it. Everyone’s experimenting with AI. But I keep hearing the same complaint: “It’s not as game-changing as I thought.” If AI is so powerful, why isn’t it doing more of your work? The #1 obstacle keeping you and your team from getting more out of AI? You're not bossing it around enough. AI doesn’t get tired and it doesn't push back. It doesn’t give you a side-eye when at 11:45 pm you demand seven rewrite options to compare while snacking in your bathrobe. Yet most people give it maybe one round of feedback—then complain it’s “meh.” The best AI users? They iterate. They refine. They make AI work for them. Here’s how: 1. Tweak AI's basic setting so it sounds like you AI-generated text can feel robotic or too formal. Fix that by teaching it your style from the start. Prompt: “Analyze the writing style below—tone, sentence structure, and word choice—and use it for all future responses.” (Paste a few of your own posts or emails.) Then, take the response and add it to Settings → Personalization → Custom Instructions. 2. Strip Out the Jargon Don’t let AI spew corporate-speak. Prompt: “Rewrite this so a smart high schooler could understand it—no buzzwords, no filler, just clear, compelling language.” or “Use human, ultra-clear language that’s straightforward and passes an AI detection test.” 3. Give It a Solid Outline AI thrives on structure. Instead of “Write me a whitepaper,” start with bullet points or a rough outline. Prompt: “Here’s my outline. Turn it into a first draft with strong examples, a compelling narrative, and clear takeaways.” Even better? Record yourself explaining your idea; paste the transcript so AI can capture your authentic voice. 4. Be Brutally Honest If the output feels off, don’t sugarcoat it. Prompt: “You’re too cheesy. Make this sound like a Fortune 500 executive wrote it.” or “Identify all weak, repetitive, or unclear text in this post and suggest stronger alternatives.” 5. Give it a tough crowd Polished isn’t enough—sometimes you need pushback. Prompt: “Pretend you’re a skeptical CFO who thinks this idea is a waste of money. Rewrite it to persuade them.” or “Act as a no-nonsense VC who doesn’t buy this pitch. Ask 5 hard questions that make me rethink my strategy.” 6. Flip the Script—AI Interviews You Sometimes the best answers come from sharper questions. Prompt: “You’re a seasoned journalist interviewing me on this topic. Ask thoughtful follow-ups to surface my best thinking.” This back-and-forth helps refine your ideas before you even start writing. The Bottom Line: AI isn’t the bottleneck—we are. If you don’t push it, you’ll keep getting mediocrity. But if you treat AI like a tireless assistant that thrives on feedback? You’ll unlock content and insights that truly move the needle. Once you work this way, there’s no going back.

Bigger context windows will not save your LLM app. Most teams think the solution is to stuff more data into the model. It is not. The real advantage comes from Context Engineering. This is the skill of designing an AI system that feeds the model the right information at the right time. Not by changing the model, but by connecting it to the outside world: • retrieving fresh data • grounding answers in facts • using tools and memory to stay accurate The goal is not to overload a prompt. It is to make the model smarter about what stays active and what gets offloaded. This is what separates basic LLM Q and A from real production systems. To do this right, you need six components working together 👇 ⸻ 1. Agents 🤖 The decision makers. Agents evaluate what they know, decide what they need, choose the right tools, and recover when things go wrong. ⸻ 2. Query Augmentation 🔎 Turning messy user input into precise intent. If the system does not know exactly what the user is asking, everything downstream fails. ⸻ 3. Retrieval 📚 The bridge from the model to your real data. This is chunking, indexing, and fetching the right facts with the right balance of precision and context. ⸻ 4. Prompting Techniques 🧭 Guiding the model with clear reasoning instructions. Chain of Thought, Few shot examples, ReAct style prompting, and more. ⸻ 5. Memory 🧠 Short term and long term. Your app needs to remember past interactions and keep persistent knowledge available when needed. ⸻ 6. Tools 🔧 The action layer. APIs, code execution, web browsing, database calls. This is how your system moves from answering questions to actually performing work. ⸻ This is far more advanced than classic RAG. This is how production systems maintain coherence, access live data, reduce hallucinations, and actually get work done. If you want more breakdowns like this on LLM architecture, RAG systems, and AI engineering, follow my profile here on LinkedIn.

What if LLMs actually get smarter with longer, more detailed instructions? New research from Stanford University & SambaNova suggests our "brevity bias" is a critical flaw in building self-improving AI. A new paper, "𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐂𝐨𝐧𝐭𝐞𝐱𝐭 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 (𝐀𝐂𝐄)," directly confronts two major issues: - Brevity Bias: Prompt optimizers often create short, generic instructions, losing vital domain-specific details. - Context Collapse: Iteratively rewriting prompts causes them to degrade over time, erasing accumulated knowledge. Instead of creating a concise summary, ACE treats context as an evolving playbook. It uses a modular workflow (Generator, Reflector, Curator) to make small, incremental "delta updates." This allows the context to grow and refine itself over time, preserving crucial details. The results: - ACE boosted performance by +10.6% on agent tasks (AppWorld) and +8.6% on complex financial analysis. - Crucially, a smaller open-source model using ACE matched the top-ranked GPT-4.1-based agent on the AppWorld leaderboard, proving the power of a superior context strategy. The implications are profound. Instead of treating prompts as static instructions, we should see them as dynamic, living knowledge bases. This paves the way for more resilient, continuously learning AI systems that adapt on the fly with incredible efficiency. #AI #MachineLearning #LLM #AIAgents #PromptEngineering

You've written the perfect prompt, but your AI agent is still failing. Why? Because the your prompt is just 5% of the input. The other 95% is the historical data, the tool definitions, the system rules is an absolute mess. We've graduated from tactical prompts. The biggest lever for AI performance today is Context Engineering and that does not mean "better RAG." It's the discipline of architecting the entire operating system for an LLM. It’s the dynamic, curated "working memory" we build for the model before it ever sees the prompt. Lazy context is why your agent loops, hallucinates, and ignores instructions. Here are 4 techniques we use internally to get our agents to perform well. 1. Your System Prompt is your AI's Constitution. Just because a prompt guru on X told you that "You are a helpful assistant. Be a lawyer" is great system prompt...its not. Define the AI's persona, rules, boundaries, and (most importantly) what it must not do. This instruction set is the most critical, persistent part of the context. Put it first. 2. Curate Memory. Sending raw unfiltered historical data won't work either. The model will suffer from the "Lost in the Middle" problem and forget key facts. Instead, engineer a memory layer: use a summarizer to create a concise "rolling summary" or a "fact sheet" of the conversation, and feed that in as context. 3. Tool Definitions Are Context. When you give an AI agents/tools via function calling, detailed tool descriptions are a critical instruction. A vague function name (e.g., "search_db") will fail. So use a precise description ("Use this function only to find a customer's order ID based on their email") is high-leverage context that controls behavior. 4. Separate and Structure All Inputs. The model needs to know what's is an "instruction," what's "interaction history," what's "retrieved data," and what's the "user query." Stop concatenating them into one messy blob. Use XML tags (<instructions>, <history_summary>, <retrieved_doc>) to create a structured information packet. If you're thinking the next 10x leap in AI will come from a 10T parameter model, it wont. It will come from organizations that master the data pipeline into the model along with architecting the entire context.

OpenAI released a guide on how to improve LLMs’ accuracy and consistency. Here are some lesser known tactics I found very interesting: 1. Prompt Baking: It involves logging the inputs and outputs during a pilot phase to identify the most effective examples. This helps you refine and prune the data into a more efficient training set, which will help improve the model's performance. 2. How to scale prompting when dealing with a long context: Having a long context can cause the LLM to struggle to maintain the attention given to all the tokens in the input context -- especially if the instructions are very complex. So, in such cases it’s important to evaluate your LLM on its ability to retrieve info from varying depths in long-context documents. Needle in A Haystack is one such model evaluation you can use. 3. Fine-Tuning with RAG Examples: They recommend incorporating your RAG context examples, into the fine-tuning process. This makes the model learn to leverage retrieved info effectively, to generate more relevant outputs. The guide also mentions common recommendations like: - Splitting complex tasks into separate calls - Using chain-of-thought prompting (you can use: https://lnkd.in/gN5eHby5) - Using GPT-4 itself to evaluate and score its outputs for iterative improvement Here's the full guide: https://lnkd.in/gAzjKdyp #AI #LLMs #OpenAI