How Language Models Use Memorization and Thought Chaining

If you’re an AI engineer trying to understand how reasoning actually works inside LLMs, this will help you connect the dots. Most large language models can generate. But reasoning models can decide. Traditional LLMs followed a straight line: Input → Predict → Output. No self-checking, no branching, no exploration. Reasoning models introduced structure, a way for models to explore multiple paths, score their own reasoning, and refine their answers. We started with Chain-of-Thought (CoT) reasoning, then extended to Tree-of-Thought (ToT) for branching, and now to Graph-based reasoning, where models connect, merge, or revisit partial thoughts before concluding. This evolution changes how LLMs solve problems. Instead of guessing the next token, they learn to search the reasoning space- exploring alternatives, evaluating confidence, and adapting dynamically. Different reasoning topologies serve different goals: • Chains for simple sequential reasoning • Trees for exploring multiple hypotheses • Graphs for revising and merging partial solutions Modern architectures (like OpenAI’s o-series reasoning models, Anthropic’s Claude reasoning stack, DeepSeek R series and DeepMind’s AlphaReasoning experiments) use this idea under the hood. They don’t just generate answers, they navigate reasoning trajectories, using adaptive depth-first or breadth-first exploration, depending on task uncertainty. Why this matters? • It reduces hallucinations by verifying intermediate steps • It improves interpretability since we can visualize reasoning paths • It boosts reliability for complex tasks like planning, coding, or tool orchestration The next phase of LLM development won’t be about more parameters, it’ll be about better reasoning architectures: topologies that can branch, score, and self-correct. I’ll be doing a deep dive on reasoning models soon on my Substack- exploring architectures, training approaches, and practical applications for engineers. If you haven’t subscribed yet, make sure you do: https://lnkd.in/dpBNr6Jg ♻️ Share this with your network 🔔 Follow along for more data science & AI insights

I’m noticing some really big shifts in how AI models handle memory. Hermes Agent got a lot of attention for its multi-layered memory and for storing how a task is done, not just what to do. And now there’s a new paper saying the same thing: it really matters to store the whole problem-solving journey. East China Normal University and others introduced Memory Intelligence Agent (MIA) framework that turns memory into something closer to experience. MIA separates the system into 3 parts: - one that stores past experiences (Manager) - one that plans how to solve a task (Planner) - and one that executes the plan (Executor) These 3 pieces evolve together and at a high level, MIA is built around a loop: retrieve memory → plan → execute → store experience → improve: 1. When a new question comes in, the system searches its memory for similar past cases, prioritizing proven successes. It also samples rarer ones to avoid being too narrow. It keeps both successful and failed examples. 2. Then the Planner takes the question and retrieved experiences and creates a step-by-step plan (like a chain-of-thought, but structured as a strategy). It keeps improving during use, while solving tasks through reinforcement learning. 3. Finally, the Executor follows this plan, interacting with tools (like search), collecting information, and trying to solve the task. This happens in a ReAct-style loop: think → act → observe → repeat. Importantly, there’s a feedback loop. After execution, the Executor reports back. If something fails or stalls, the Planner reflects, updates the plan, and tries again, only when needed. Another important part of MIA is compression of the experience: long trajectories become structured summaries of workflows, images become captions, and redundant memory is replaced or merged. And it turns out that MIA uses 2 types of memory at the same time: - Non-parametric memory → explicit storage (retrieved examples, workflows) - Parametric memory → knowledge inside the model (learned weights) Results clearly show the advantages of MIA's workflow: • It becomes new SOTA among memory agents: +5.5 avg gain, ~53.6 accuracy • Achieves up to +9.1 on complex tasks like multi-hop/in-house reasoning • Small 7B model matches / beats larger closed models • MIA is also strong on multimodal & text tasks and improves further via test-time learning Yes, this line of research definitely needs to continue, because the question of memory efficiency in agents is only just beginning to mature.

Chain-of-Thought: The Pattern that Taught AI to Think Before Speaking Large Language Models are naturally designed to jump straight to an answer without much forethought. This is because they are fundamentally built to predict the next likely word based on all of the text they were trained with, rather than to perform actual reasoning. If you ask an LLM to write a software script, it could actually invent a library function that sounds plausible but doesn't actually exist, just to make the code look complete. That’s impressive and scary at the same time. That also led to the need for Chain-of-Thought, a design pattern that prevents this by forcing the AI system to pause and break a problem down into logical steps *before* providing its final answer. This technique became an overnight sensation in the AI world when researchers discovered that adding the simple magic phrase "let's think step-by-step" stopped a model from acting impulsively and actually helped it solve complex problems. For developers, implementing this is as simple as adding a system prompt that demands that the AI "show its work" alongside its final answer. While this does make a model slower and more resource-hungry, it is now considered a backbone of agentic AI. Autonomous agents simply cannot navigate unpredictable real-world workflows without this type of structured reasoning. But, just keep in mind that while Chain-of-Thought produces a plan, it’s still just outputting text. You will need to add other design patterns to get the agent to stop talking and actually do the work. Many of the other design patterns I’ll be covering in future posts are designed to take the baton from Chain-of-Thought to put those plans into action. Whether you’re a student or a practitioner, I’d love to hear more about your experiences with this core design pattern. Be sure to share your thoughts in the comments. This design pattern is covered in the following video of my LinkedIn Learning course “Agentic AI Solution Design Patterns”: https://lnkd.in/g4iAiXm5 To learn more about Arcitura’s AI & Cloud AI Professional Academy courses and certification programs, see here: https://lnkd.in/gXRzqmcq #AgenticAI #ChainOfThought #LLM #AIReasoning #DesignPatterns #AI #LinkedInLearning #Arcitura

WTF is the deal with DeepSeek-R1? Here's my take for non-AI folks – We have seen that LLMs perform better on hard tasks like math if we teach them to "think about" the problem first. This technique is called "chain-of-thought" (CoT). The language model is taught to emit sentences that break a problem down before answering it. OpenAI's o1 does this and performs well because of it. Training this behavior needs many examples of correct chains of thought. These are expensive to produce and expensive to train on. DeepSeek discovered something surprising. It turns out, you don’t need to explicitly train a model to produce a chain of thought. Instead, under the right conditions, models will learn this behavior emergently. They found a way for a language model to learn chain-of-thought cheaply, and released that model as open source. We now have an o1-quality LLM that's 27x cheaper to use. That's a big deal on it's own. But DeepSeek also released their "recipe" to let developers teach chain-of-thought to other models. Thought chains are extremely useful. Now that they're cheap and easy to produce, we are discovering all the different ways they can be used. These are some of the open questions being explored right now: Can we teach small models to emergently learn chain-of-thought? Yes! ↳ How small? ↳ How cheaply? ↳ On which tasks? Can we generate thought chains with R1 and just copy/paste them into the prompts of other models? Yes! ↳ Why does this work? ↳ What problems does this work for? ↳ Does it generalize? Why does this behavior emerge? When should we use RL vs SFT? How many GPUs does DeepSeek have, actually? In summary: we found a way to make LLMs show their work without spending a fortune. Now it's time to figure out what we can build with this.