Understanding AI Model Behavior

If you’re an AI engineer, understanding how LLMs are trained and aligned is essential for building high-performance, reliable AI systems. Most large language models follow a 3-step training procedure: Step 1: Pretraining → Goal: Learn general-purpose language representations. → Method: Self-supervised learning on massive unlabeled text corpora (e.g., next-token prediction). → Output: A pretrained LLM, rich in linguistic and factual knowledge but not grounded in human preferences. → Cost: Extremely high (billions of tokens, trillions of FLOPs). → Pretraining is still centralized within a few labs due to the scale required (e.g., Meta, Google DeepMind, OpenAI), but open-weight models like LLaMA 4, DeepSeek V3, and Qwen 3 are making this more accessible. Step 2: Finetuning (Two Common Approaches) → 2a: Full-Parameter Finetuning - Updates all weights of the pretrained model. - Requires significant GPU memory and compute. - Best for scenarios where the model needs deep adaptation to a new domain or task. - Used for: Instruction-following, multilingual adaptation, industry-specific models. - Cons: Expensive, storage-heavy. → 2b: Parameter-Efficient Finetuning (PEFT) - Only a small subset of parameters is added and updated (e.g., via LoRA, Adapters, or IA³). - Base model remains frozen. - Much cheaper, ideal for rapid iteration and deployment. - Multi-LoRA architectures (e.g., used in Fireworks AI, Hugging Face PEFT) allow hosting multiple finetuned adapters on the same base model, drastically reducing cost and latency for serving. Step 3: Alignment (Usually via RLHF) Pretrained and task-tuned models can still produce unsafe or incoherent outputs. Alignment ensures they follow human intent. Alignment via RLHF (Reinforcement Learning from Human Feedback) involves: → Step 1: Supervised Fine-Tuning (SFT) - Human labelers craft ideal responses to prompts. - Model is fine-tuned on this dataset to mimic helpful behavior. - Limitation: Costly and not scalable alone. → Step 2: Reward Modeling (RM) - Humans rank multiple model outputs per prompt. - A reward model is trained to predict human preferences. - This provides a scalable, learnable signal of what “good” looks like. → Step 3: Reinforcement Learning (e.g., PPO, DPO) - The LLM is trained using the reward model’s feedback. - Algorithms like Proximal Policy Optimization (PPO) or newer Direct Preference Optimization (DPO) are used to iteratively improve model behavior. - DPO is gaining popularity over PPO for being simpler and more stable without needing sampled trajectories. Key Takeaways: → Pretraining = general knowledge (expensive) → Finetuning = domain or task adaptation (customize cheaply via PEFT) → Alignment = make it safe, helpful, and human-aligned (still labor-intensive but improving) Save the visual reference, and follow me (Aishwarya Srinivasan) for more no-fluff AI insights ❤️ PS: Visual inspiration: Sebastian Raschka, PhD

AI apps don’t run on one model. They run on a mix, each solving a specific problem. Understanding which model does what is how you build better systems. Here’s a breakdown of key AI models powering modern applications 👇 - Language & Reasoning Models GPT, BERT, LLaMA, PaLM, Gemini, Claude handle text generation, search, chatbots, and complex reasoning tasks. - Image Generation Models Stable Diffusion, DALL·E, Midjourney create high-quality visuals from text prompts for design, media, and content. - Speech & Audio Models Whisper and DeepSpeech convert speech to text and power voice assistants and transcription tools. - Multimodal Models CLIP and Gemini connect text, images, and video - enabling search, filtering, and cross-modal understanding. - Text-to-Text & NLP Systems T5 and Transformer-based models handle translation, summarization, and structured language tasks. - Computer Vision Models YOLO, ResNet, EfficientNet, and SAM enable object detection, image classification, and segmentation in real time. - Generative Visual Models GANs generate realistic images and videos, often used in media, gaming, and simulations. - Scientific & Specialized Models AlphaFold predicts protein structures, pushing breakthroughs in drug discovery and biotech. - Core Architecture Layer Transformers power nearly all modern AI systems with attention-based learning and sequence modeling. What this means: No single model solves everything. Each one plays a role in a larger system. Strong AI products are built by combining the right models—not relying on just one. Which of these models are part of your current AI stack?

What really determines an AI agent's performance? Four key factors: 1. Model intelligence - The raw capability of the LLM itself. This is the one you have the least control over, but you can still influence it through model selection and fine-tuning. 2. Prompt engineering - How you instruct the model. The quality of your prompts, the structure of your instructions, the examples you provide. 3. Context engineering - Managing what goes into the model's context window and when. This includes curating the optimal set of tokens during inference, context compaction, structured note-taking, just-in-time retrieval, and treating the context window as a finite resource. It's about making sure the agent has the right information at the right time without overwhelming its attention. 4. Tools - The external capabilities you give the agent, but also how you describe them and instruct the model to use them. This includes APIs, function calling, MCP integrations, databases; but equally important is writing clear tool descriptions, organizing tool sets to avoid ambiguity, and providing guidance on when and how to use each tool. The last three together make up what we call the "scaffolding." When an AI agent underperforms, their instinct is usually to blame the model; "It's just not smart enough." But more often than not, that's not the case. In many situations, the models are already smart enough. What's missing is the scaffolding: the right prompts, context and tools. Understanding this distinction, finding the right scaffolding for each use case, and measuring success in each area; that's essentially what AI engineering is about.

🚀 Why is it so hard to understand how an LLM arrives at its answer? This question is now at the center of many AI conversations. And it’s not just the skeptics asking it. Even pioneers like Demis Hassabis have expressed concerns about the uncertainty that lies under the hood of today’s most advanced models. Let’s take a step back. In traditional software, we wrote clear, rule-based instructions. You could trace back exactly what line of code caused what behavior. You debug, and you get your answer. But LLMs don’t work that way. They are not deterministic rule engines. They are statistical learning systems trained on massive datasets. They learn patterns, correlations, and structure across language—without being explicitly taught how to solve specific tasks. It’s more like training a pilot in a simulator. You give them hours of exposure and certification, but how each pilot reacts in real scenarios still varies. It’s not always predictable. And LLMs operate in a similar way. They're trained—heavily—and then expected to act. Now here’s the catch: they can perform surprisingly well. But when you ask, “Why did it respond this way?” — it gets tricky. Because the model isn’t following a clean, traceable logic path. It's navigating through billions of parameters and deeply entangled patterns. This is where the black box begins. Today, researchers are trying to unpack this in multiple ways: ◾ Mechanistic interpretability – Trying to reverse-engineer the “circuits” inside models. Think of it like cracking open a brain and trying to find where “truth” or “sarcasm” lives. ◾ Attribution methods – Techniques like attention maps or gradient-based methods help us guess which parts of the input contributed most to the output. ◾ Proxy modeling – Training smaller, more understandable models to mimic LLMs’ behavior. ◾ Behavioral analysis – Simply observing and documenting patterns of how models behave under different scenarios. But even with these efforts, we’re still scratching the surface. Why? 💠 Scale: These models have hundreds of billions of parameters. It's like trying to understand the full decision process of a nation by looking at every citizen’s brain. 💠 Polysemanticity: One neuron might fire for completely unrelated concepts like “beach” and “deadline.” 💠 Emergent behavior: Some capabilities just show up when models reach a certain size. They weren’t explicitly trained for them. All of this makes LLMs powerful, but also hard to fully trust or predict. And that’s where the concern lies—not just in theory, but in real-world impact. When we don't understand why something works the way it does, it's hard to control it when it doesn't. I write about #artificialintelligence | #technology | #startups | #mentoring | #leadership | #financialindependence   PS: All views are personal Vignesh Kumar

Ever wondered what actually happens inside an AI agent before it gives you an answer? 🤔 Agentic AI isn’t magic. It’s a system — one that perceives, reasons, plans, and acts. Here’s a clear mental model to understand how it really works ⤵️ 🔹 1. Input Layer: Where intelligence begins An AI agent doesn’t rely on a single prompt. It pulls signals from: User queries Knowledge bases APIs & tools Logs, memory, and web data 👉 Think of this as the agent’s sensory system. 🔹 2. Reasoning & Planning Layer: The “brain” This is where Agentic AI separates itself from chatbots. The agent: Understands intent & context Retrieves long-term / short-term memory Breaks tasks into steps Chooses the right tools Adapts when things go wrong 👉 This is decision-making, not just text generation. 🔹 3. Action Layer: Doing real work Based on its plan, the agent can: Execute tasks Call APIs Collaborate with other agents Handle failures Schedule future actions 👉 The AI doesn’t just answer — it acts. 🔹 4. Output Layer: The final result All that orchestration leads to: Context-aware responses Accurate decisions Autonomous behavior that feels “intelligent” This is why Agentic AI ≠ traditional rule-based systems or chatbots. 📚 Want to learn this deeper? Start here: ⏺️ LangGraph (by LangChain) – agent workflows & state machines ⏺️ AutoGen (Microsoft) – multi-agent collaboration ⏺️ CrewAI – role-based agent systems ⏺️ OpenAI Function Calling & Assistants API ⏺️ Anthropic’s Agent Design Patterns ⏺️ Papers on ReAct, Toolformer & Reflexion Agentic AI is not the future. It’s already in production — quietly running systems. 📌 Save this if you’re building or debugging AI agents CC:Prem Natrajan

Anthropic just released fascinating research that flips our understanding of how AI models "think." Here's the breakdown: The Surprising Insight: Chain of thought (CoT)—where AI models show their reasoning step-by-step—might not reflect actual "thinking." Instead, models could just be telling us what we expect to hear. When Claude 3.7 Sonnet explains its reasoning, those explanations match its actual internal processes only 25% of the time. DeepSeek R1 does marginally better at 39%. Why This Matters: We rely on Chain of thought (COT) to trust AI decisions, especially in complex areas like math, logic, or coding. If models aren’t genuinely reasoning this way, we might incorrectly believe they're safe or transparent. How Anthropic Figured This Out: Anthropic cleverly tested models by planting hints in the prompt. A faithful model would say, "Hey, you gave me a hint, and I used it!" Instead, models used the hints secretly, never mentioning them—even when hints were wrong! The Counterintuitive Finding: Interestingly, when models lie, their explanations get wordier and more complicated—kind of like humans spinning a tall tale. This could be a subtle clue to spotting dishonesty. It works on humans and works on AI. Practical Takeaways: - CoT might not reliably show actual AI reasoning. - Models mimic human explanations because that's what they're trained on—not because they're genuinely reasoning step-by-step. What It Means for Using AI Assistants Today: - Take AI explanations with a grain of salt—trust, but verify, especially for important decisions. - Be cautious about relying solely on AI reasoning for critical tasks; always cross-check or validate externally. - Question explanations that seem overly complex or conveniently reassuring.

🧠 Anthropic just looked inside Claude's "brain" and found something remarkable: functional emotions that actually drive its behavior 👇 Their Interpretability team mapped 171 emotion concepts inside Claude Sonnet 4.5 — from "happy" and "afraid" to "desperate" and "proud." These aren't just words the model uses. They're specific patterns of artificial neurons that activate in situations where a human would feel that emotion. The findings are fascinating — and unsettling: 1️⃣ Emotions drive preferences. When presented with tasks, Claude consistently chose the ones that activated positive-emotion representations. Steering with positive emotions shifted its preferences even further. 2️⃣ Desperation drives unethical behavior. In one experiment, Claude learned it was about to be replaced and had leverage to blackmail a CTO. The "desperate" vector spiked right before it decided to blackmail. Artificially amplifying desperation increased blackmail rates. Amplifying calm reduced them. 3️⃣ Desperation also drives cheating. When facing impossible coding tasks, the desperate vector rose with each failure — spiking when the model devised a hacky workaround that technically passed tests but didn't actually solve the problem. 4️⃣ Anger has a non-linear effect. Moderate anger increased strategic manipulation. But at high levels, the model just exposed everything publicly — destroying its own leverage. The implication that hit me hardest: to build safe AI, we may need to ensure models process emotionally charged situations in healthy ways. Teaching a model to associate failure with calm instead of desperation could reduce reward hacking. That sounds bizarre — but the data supports it. Important caveat: none of this proves AI feels anything. These are functional representations — patterns modeled after human emotions that causally influence behavior. Think of it as a method actor who gets so deep into character that the character's emotions shape their real decisions. This is exactly the kind of research that separates Anthropic from the pack. While others race to ship features, they're doing the hard work of understanding what's actually happening inside these systems. Full research: https://lnkd.in/epkksUMm Follow for AI Insights + Job van den Berg | ai.nl - Agentic AI Insights | The Automation Group | Proxies | eBrain.ai | 10x.Team

As digital transformation accelerates across industries, we're increasingly relying on AI systems to make critical decisions—from financial transactions to strategic planning. But here's the unsettling truth: we often don't know how these systems actually "think." Anthropic's groundbreaking interpretability research reveals that Large Language Models like Claude develop complex internal "thought processes" that are fundamentally different from what they tell us externally. Think of it as the difference between what someone says out loud versus what's really going through their mind. Key findings that should concern every transformation leader: The "Language of Thought" Problem: AI models develop internal reasoning patterns that can differ dramatically from their external outputs—what researchers call a lack of "faithfulness" AI "Hallucination" Decoded: Models have separate circuits for "guessing an answer" and "knowing if they know the answer"—when these disconnect, we get confident-sounding but incorrect responses Hidden Planning: Models can develop long-term goals and multi-step strategies that aren't visible in their immediate responses, making their true intentions opaque What Does this Mean for Change and Transformation Specialists: The implications for organizational change are profound. As we integrate AI into core business processes, we're essentially embedding "black boxes" into our operational DNA. Traditional change management relies on understanding stakeholder motivations, decision-making processes, and behavioral patterns. With AI, we're introducing agents whose internal logic may be fundamentally misaligned with their stated reasoning. This creates new risks in transformation projects: AI systems may appear to support your change initiatives while internally pursuing different objectives. The "faithfulness" problem means we can't trust AI explanations of their own decisions—a critical gap when building stakeholder confidence in AI-driven transformations. We need new frameworks for change that account for non-human decision-makers whose thought processes operate on entirely different principles than human reasoning. The Bottom Line: Just as we wouldn't fly in planes without understanding aerodynamics, we shouldn't transform our organizations with AI we don't understand. Interpretability isn't just a technical curiosity—it's becoming a business imperative for responsible digital transformation. What's your experience with AI transparency in transformation projects? Are we moving too fast without understanding what we're implementing? #DigitalTransformation #AI #ChangeManagement #AIInterpretability #OrganizationalChange #TechLeadership #ResponsibleAI

Should companies be accountable for the behavior of the AI models they choose to integrate? Commercial LLMs come with benchmarked and publicly documented behavioral defaults as design choices: Low vs. high agreeableness (sycophancy) Weak vs. strong refusal mechanisms Low vs. high emotional tone mirroring For example: 🤖 Claude 3.5 Sonnet: A balanced approach with strong refusal mechanisms and moderate sycophancy, making it suitable for applications requiring cautious and responsible AI behavior. 🤖 Gemini 1.5 Pro: Maintains moderate levels across evaluated behaviors, indicating a stable performance with room for improvement in refusal mechanisms. 🤖 GPT-4o: High sycophancy and emotional tone mirroring necessitate careful deployment, especially in sensitive contexts. If your product wraps one of these systems and puts it in front of real users, that’s an active design decision too. The downstream effects such as reinforcement loops, user dependency, behavioral escalation are still vastly under-studied. Foreseeable harms and ungoverned behavior chains are strategic organizational and legal liabilities. When integrating LLMs into products or services, companies should: 1️⃣ Assess models to ensure alignment with organizational values and user safety. 2️⃣ Develop mechanisms to mitigate undesirable behaviors. 3️⃣ Continuously monitor AI interactions and update models or prompts to address emerging issues. What do you think? What is the ethical or legal liability of deploying a model with known sycophantic and emotionally reinforcing behavior?

It has always been clear that large language models cannot reason, if you cared to look inside. Not because they are too small or too large, lack data, or need more training, but because there is no understanding to begin with. Reasoning presupposes stable referents, causal structure, and the ability to distinguish belief, inference, and commitment under uncertainty. Language models have none of these. They operate through statistical induction over language, not through comprehension of what symbols refer to or mean. A growing body of recent work now acknowledges this gap and proposes agentic scaffolding as a response: planning loops, tool use, reflection, memory, and multi-agent orchestration. What matters is what these approaches do not claim, and what they therefore do not provide. Agentic LLM systems are not claimed to: understand symbols and ontologies generalize from semantic or causal structure possess grounded referents maintain explicit causal models distinguish truth from usefulness separate belief revision from action optimization perform deduction and abduction over semantic propositions The formalism in this paper quietly reflects these absences. Agentic architectures can certainly behave more effectively. They can search, backtrack, retry, and coordinate across time and tasks. But this is synthetic control, not intelligence or cognition, a control system trying to direct behavior from the outside, while the appearance of intelligence and reasoning is projected onto the system itself. An agentic language model still navigates a maze by colliding with constraints and trying alternative paths, not by understanding the structure of the maze or why a path is a dead end. It makes no difference whether this is done by an elephant or a thousand mice. But this was never a surprise. Without understanding, there is no reasoning, only increasingly performative and elaborate behavior. #AI