How to Measure LLM Intelligence

New Google paper challenges how we measure LLM reasoning. Token count is a poor proxy for actual reasoning quality. There might be a better way to measure this. This work introduces "deep-thinking tokens," a metric that identifies tokens where internal model predictions shift significantly across deeper layers before stabilizing. These tokens capture "genuine reasoning" effort rather than verbose output. Instead of measuring how much a model writes, measure how hard it's actually thinking at each step. Deep-thinking tokens are identified by tracking prediction instability across transformer layers during inference. The ratio of deep-thinking tokens correlates more reliably with accuracy than token count or confidence metrics across mathematical and scientific benchmarks (AIME 24/25, HMMT 25, GPQA-diamond), tested on DeepSeek-R1, Qwen3, and GPT-OSS. They also introduce Think@n, a test-time compute strategy that prioritizes samples with high deep-thinking ratios while early-rejecting low-quality partial outputs, reducing cost without sacrificing performance. Why does it matter? As inference-time scaling becomes a primary lever for improving model performance, we need better signals than token length to understand when a model is actually reasoning versus just rambling.

Evaluating LLMs is hard. Evaluating agents is even harder. This is one of the most common challenges I see when teams move from using LLMs in isolation to deploying agents that act over time, use tools, interact with APIs, and coordinate across roles. These systems make a series of decisions, not just a single prediction. As a result, success or failure depends on more than whether the final answer is correct. Despite this, many teams still rely on basic task success metrics or manual reviews. Some build internal evaluation dashboards, but most of these efforts are narrowly scoped and miss the bigger picture. Observability tools exist, but they are not enough on their own. Google’s ADK telemetry provides traces of tool use and reasoning chains. LangSmith gives structured logging for LangChain-based workflows. Frameworks like CrewAI, AutoGen, and OpenAgents expose role-specific actions and memory updates. These are helpful for debugging, but they do not tell you how well the agent performed across dimensions like coordination, learning, or adaptability. Two recent research directions offer much-needed structure. One proposes breaking down agent evaluation into behavioral components like plan quality, adaptability, and inter-agent coordination. Another argues for longitudinal tracking, focusing on how agents evolve over time, whether they drift or stabilize, and whether they generalize or forget. If you are evaluating agents today, here are the most important criteria to measure: • 𝗧𝗮𝘀𝗸 𝘀𝘂𝗰𝗰𝗲𝘀𝘀: Did the agent complete the task, and was the outcome verifiable? • 𝗣𝗹𝗮𝗻 𝗾𝘂𝗮𝗹𝗶𝘁𝘆: Was the initial strategy reasonable and efficient? • 𝗔𝗱𝗮𝗽𝘁𝗮𝘁𝗶𝗼𝗻: Did the agent handle tool failures, retry intelligently, or escalate when needed? • 𝗠𝗲𝗺𝗼𝗿𝘆 𝘂𝘀𝗮𝗴𝗲: Was memory referenced meaningfully, or ignored? • 𝗖𝗼𝗼𝗿𝗱𝗶𝗻𝗮𝘁𝗶𝗼𝗻 (𝗳𝗼𝗿 𝗺𝘂𝗹𝘁𝗶-𝗮𝗴𝗲𝗻𝘁 𝘀𝘆𝘀𝘁𝗲𝗺𝘀): Did agents delegate, share information, and avoid redundancy? • 𝗦𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗼𝘃𝗲𝗿 𝘁𝗶𝗺𝗲: Did behavior remain consistent across runs or drift unpredictably? For adaptive agents or those in production, this becomes even more critical. Evaluation systems should be time-aware, tracking changes in behavior, error rates, and success patterns over time. Static accuracy alone will not explain why an agent performs well one day and fails the next. Structured evaluation is not just about dashboards. It is the foundation for improving agent design. Without clear signals, you cannot diagnose whether failure came from the LLM, the plan, the tool, or the orchestration logic. If your agents are planning, adapting, or coordinating across steps or roles, now is the time to move past simple correctness checks and build a robust, multi-dimensional evaluation framework. It is the only way to scale intelligent behavior with confidence.

Evaluating LLMs accurately/reliably is difficult, but we can usually automate the evaluation process with another (more powerful) LLM... Automatic metrics: Previously, generative text models were most commonly evaluated using automatic metrics like ROUGE and BLEU, which simply compare how well a model’s output matches a human-written target resopnse. In particular, BLEU score was commonly used to evaluatate machine translation models, while ROUGE was most often used for evaluating summarization models. Serious limitations: With modern LLMs, researchers began to notice that automatic metrics did a poor job of comprehensively capturing the quality of an LLM’s generations. Oftentimes, ROUGE scores were poorly correlated with human preferences—higher scores don’t seem to indicate a better generation/summary [1]. This problem is largely due to the open-ended nature of most tasks solved with LLMs. There can be many good responses to a prompt. LLM-as-a-judge [2] leverages a powerful LLM (e.g., GPT-4) to evaluate the quality of an LLM’s output. To evaluate an LLM with another LLM, there are three basic structures or strategies that we can employ: (1) Pairwise comparison: The LLM is shown a question with two responses and asked to choose the better response (or declare a tie). This approach was heavily utilized by models like Alpaca/Vicuna to evaluate model performance relative to proprietary LLMS like ChatGPT. (2) Single-answer grading: The LLM is shown a response with a single answer and asked to provide a score for the answer. This strategy is less reliable than pairwise comparison due to the need to assign an absolute score to the response. However, authors in [2] observe that GPT-4 can nonetheless assign relatively reliable/meaningful scores to responses. (3) Reference-guided grading: The LLM is provided a reference answer to the problem when being asked to grade a response. This strategy is useful for complex problems (e.g., reasoning or math) in which even GPT-4 may struggle with generating a correct answer. In these cases, having direct access to a correct response may aid the grading process. “LLM-as-a-judge offers two key benefits: scalability and explainability. It reduces the need for human involvement, enabling scalable benchmarks and fast iterations.” - from [2] Using MT-bench, authors in [2] evaluate the level of agreement between LLM-as-a-judge and humans (58 expert human annotators), where we see that there is a high level of agreement between these strategies. Such a finding caused this evaluation strategy to become incredibly popular for LLMs—it is currently the most widely-used and effective alternative to human evaluation. However, LLM-as-a-judge does suffer from notable limititations (e.g., position bias, verbosity bias, self-enhancement bias, etc.) that should be considered when interpretting data.

Stop debating if your AI is “good.” Measure it. LLM-as-a-Judge is how operators do it at scale. It turns fuzzy reviews into consistent scores. So you can ship, improve, and prove ROI. What it is: an LLM that grades other outputs. When to use: subjective, multi-criteria, high volume. When not to: clear ground truth or legal high-stakes. Three ways to score, pick one: ✅Single-output with a reference for grounding. ✅Single-output without a reference for style fit. ✅Pairwise comparisons for ranking variants. Bias is real. Plan for it. Length, order, authority, and self-favor creep in. Mitigate with controls, not vibes. ✅ Randomize candidate order. ✅ Cap and normalize length. ✅ Hide sources and identities. ✅ Use 3 judges and average. Make the judge predictable. Write criteria in plain language. Force a strict JSON schema for scores. Reject outputs that break the schema. Require a brief rationale with evidence. Then validate the validator. Test on easy, tricky, and adversarial cases. Track precision, recall, AUROC, and agreement. Run it next to humans and compare. Scale without breaking the bank. Use a small evaluation model for real-time checks. Spot-audit with a larger model weekly. Operators: start with one workflow this week. Ship the judge, log every decision, improve weekly. Save this and share with one teammate who owns QA.

LLM Evaluation: Why Testing AI Models Is No Longer Optional Organizations are deploying LLMs at an incredible pace—often treating them like high-performing employees who can execute tasks instantly. But here’s the uncomfortable question: Are we actually checking their work? Without rigorous evaluation, speed can easily mask hidden risks—hallucinations, bias, reasoning gaps, and unreliable outputs. LLM evaluation is essentially quality control for AI. It helps us: • Measure performance against ground truth • Identify blind spots and knowledge gaps • Detect bias and harmful outputs • Compare models using standardized benchmarks • Build trust with users and stakeholders In enterprise environments—especially regulated sectors like finance, healthcare, and public sector—evaluation isn’t just a best practice. It’s a governance requirement. Metrics like accuracy, recall, F1, coherence, latency, toxicity, BLEU, and ROUGE give us a multi-dimensional view of model behavior—not just “does it sound good?” Frameworks such as MMLU, HumanEval, TruthfulQA, GLUE, and IBM FM-Eval are becoming foundational to LLMOps and responsible AI programs. The real shift happening right now: AI is moving from experimentation → operational infrastructure And infrastructure must be measurable, auditable, and reliable. #AI #GenerativeAI #LLMOps #ResponsibleAI #AIGovernance #AIEngineering #EnterpriseAI #AgenticAI Image Credit : The Gen Academy

Evaluation is an exciting space and so critical to put AI apps in production and helping them perform same or better as environment or models change. Still looking for a great company in this space Four primary evaluation methodologies, which can be broadly categorized as either benchmark-based or judgment-based. The four core methods are: 1. Multiple-Choice Benchmarks: Quantify an LLM's knowledge recall through standardized tests like MMLU. They are reproducible and scalable but do not assess real-world utility or reasoning. 2. Verifiers: Assess free-form answers in domains like math and code by programmatically checking a final, extracted answer against a ground truth. This is crucial for evaluating reasoning but is limited to deterministically verifiable domains. 3. Leaderboards: Rank models based on aggregated human preferences, as exemplified by LM Arena. This method captures subjective qualities like style and helpfulness but is susceptible to bias and lacks the instant feedback needed for active development. 4. LLM-as-a-Judge: Employ a powerful LLM to score another model's output against a reference answer using a detailed rubric. This is a scalable and consistent alternative to human evaluation but is highly dependent on the judge model's capabilities and the rubric's design. A strong score in multiple-choice benchmarks suggests solid general knowledge. High performance on verifier-based tasks indicates proficiency in technical domains. However, if that same model scores poorly on leaderboards or LLM-as-a-judge evaluations, it may indicate issues with articulation, style, or user helpfulness, suggesting a need for fine-tuning. https://lnkd.in/gSdFpScW

How do we actually evaluate LLMs? A simple question with a long answer! I wrote a new article explaining the four main approaches: 1. Multiple-choice benchmarks 2. Verifiers 3. Leaderboards 4. and LLM-as-a-judge methods Each one with hands-on, from-scratch code examples so you can see how they work under the hood and what the trade-offs are. Of course, there is no single "best" method; each has its trade-offs and its place depending on the goal. But understanding these foundations hopefully helps make sense of all the leaderboards, papers, and various claims we see every week (/day). 🔗 https://lnkd.in/gHCmmzas

In the rapidly evolving world of conversational AI, Large Language Model (LLM) based chatbots have become indispensable across industries, powering everything from customer support to virtual assistants. However, evaluating their effectiveness is no simple task, as human language is inherently complex, ambiguous, and context-dependent. In a recent blog post, Microsoft's Data Science team outlined key performance metrics designed to assess chatbot performance comprehensively. Chatbot evaluation can be broadly categorized into two key areas: search performance and LLM-specific metrics. On the search front, one critical factor is retrieval stability, which ensures that slight variations in user input do not drastically change the chatbot's search results. Another vital aspect is search relevance, which can be measured through multiple approaches, such as comparing chatbot responses against a ground truth dataset or conducting A/B tests to evaluate how well the retrieved information aligns with user intent. Beyond search performance, chatbot evaluation must also account for LLM-specific metrics, which focus on how well the model generates responses. These include: - Task Completion: Measures the chatbot's ability to accurately interpret and fulfill user requests. A high-performing chatbot should successfully execute tasks, such as setting reminders or providing step-by-step instructions. - Intelligence: Assesses coherence, contextual awareness, and the depth of responses. A chatbot should go beyond surface-level answers and demonstrate reasoning and adaptability. - Relevance: Evaluate whether the chatbot’s responses are appropriate, clear, and aligned with user expectations in terms of tone, clarity, and courtesy. - Hallucination: Ensures that the chatbot’s responses are factually accurate and grounded in reliable data, minimizing misinformation and misleading statements. Effectively evaluating LLM-based chatbots requires a holistic, multi-dimensional approach that integrates search performance and LLM-generated response quality. By considering these diverse metrics, developers can refine chatbot behavior, enhance user interactions, and build AI-driven conversational systems that are not only intelligent but also reliable and trustworthy. #DataScience #MachineLearning #LLM #Evaluation #Metrics #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gj6aPBBY    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/gAC8eXmy

Evaluating ML is easy. Use metrics like AUC or MSE. But what about LLMs? ↓ LLM evaluation is not easy. Unless the task is a simple classification like flagging an email as ham or spam, it's difficult since... ☒ Manual review is costly ☒ Task input/output is open-ended ☒ Benchmarks like MMLU generic for custom cases So, how do you evaluate on a scale? Here are 3 strategies to employ ↓ 𝟭. 𝗦𝗲𝗺𝗮𝗻𝘁𝗶𝗰 𝗦𝗶𝗺𝗶𝗹𝗮𝗿𝗶𝘁𝘆 Two texts with similar meanings will have embedding vectors that are close together. Use cosine similarity to compare ideal output samples with LLM-generated responses. A higher score indicates a better response. 𝟮. 𝗟𝗟𝗠-𝗮𝘀-𝗮-𝗝𝘂𝗱𝗴𝗲 Getting a human to evaluate LLM output is costly. So, create an LLM agent that mimics a human reviewer. Create a prompt with a grading rubric with examples. Get the reviewer agent to evaluate the main agent on a scale. 𝟯. 𝗘𝘅𝗽𝗹𝗶𝗰𝗶𝘁 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸 Add a UI to the chat interface to gather thumbs up/down and re-generation feedback. This helps measure the quality of the output from the users themselves. With this feedback loop in place, optimize your LLM system with prompt engineering, fine-tuning, RAG, and other techniques. Let's bounce ideas around. How do you evaluate LLM? Drop a comment ↓

Here's the LLM evaluation stack I recommend to every team: Layer 1: Unit Tests (DeepEval) Stop treating AI as a mystery box. Integrate with Pytest to run assertions on every build. → Test individual components (retrievers, generators, tools) → Run in CI/CD to block regressions → Move from vibe-checking to deterministic engineering Layer 2: Metric Suite (50+ SOTA Metrics) Quantify performance with academic-grade metrics, not just "looks good" scores: → Hallucination: Is it making things up? → Faithfulness: Is it strictly grounded in your context? → Agentic Trajectory: Did it pick the right tool and use the correct arguments? → G-Eval: Define custom, subjective criteria in plain English. Layer 3: Synthetic Data Evolution Don't wait for user logs to find your bugs. → Generate thousands of "Golden" test cases from your docs in minutes → Automatically cover complex edge cases → Scale your testing without a single manual label Layer 4: Continuous Monitoring Evaluation doesn't stop at deployment. → Track performance drift in real-time → Get a "Rationale" (the why) for every production failure → A/B test prompt versions with statistical confidence DeepEval handles all 4 layers in one framework. One framework: ✓ 50+ research-backed metrics ✓ Pytest-native syntax ✓ Synthetic data generation ✓ Full Agent & RAG support This is how you ship AI with actual confidence. (100% Open-Source) GitHub Repo - https://lnkd.in/gQ3zCcZN Don't forget to ⭐️