Evaluating AI-Generated Content With LLMs

Evaluations —or “Evals”— are the backbone for creating production-ready GenAI applications. Over the past year, we’ve built LLM-powered solutions for our customers and connected with AI leaders, uncovering a common struggle: the lack of clear, pluggable evaluation frameworks. If you’ve ever been stuck wondering how to evaluate your LLM effectively, today's post is for you. Here’s what I’ve learned about creating impactful Evals: 𝗪𝗵𝗮𝘁 𝗠𝗮𝗸𝗲𝘀 𝗮 𝗚𝗿𝗲𝗮𝘁 𝗘𝘃𝗮𝗹? - Clarity and Focus: Prioritize a few interpretable metrics that align closely with your application’s most important outcomes. - Efficiency: Opt for automated, fast-to-compute metrics to streamline iterative testing. - Representation Matters: Use datasets that reflect real-world diversity to ensure reliability and scalability. 𝗧𝗵𝗲 𝗘𝘃𝗼𝗹𝘂𝘁𝗶𝗼𝗻 𝗼𝗳 𝗠𝗲𝘁𝗿𝗶𝗰𝘀: 𝗙𝗿𝗼𝗺 𝗕𝗟𝗘𝗨 𝘁𝗼 𝗟𝗟𝗠-𝗔𝘀𝘀𝗶𝘀𝘁𝗲𝗱 𝗘𝘃𝗮𝗹𝘀 Traditional metrics like BLEU and ROUGE paved the way but often miss nuances like tone or semantics. LLM-assisted Evals (e.g., GPTScore, LLM-Eval) now leverage AI to evaluate itself, achieving up to 80% agreement with human judgments. Combining machine feedback with human evaluators provides a balanced and effective assessment framework. 𝗙𝗿𝗼𝗺 𝗧𝗵𝗲𝗼𝗿𝘆 𝘁𝗼 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗲: 𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗬𝗼𝘂𝗿 𝗘𝘃𝗮𝗹 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲 - Create a Golden Test Set: Use tools like Langchain or RAGAS to simulate real-world conditions. - Grade Effectively: Leverage libraries like TruLens or Llama-Index for hybrid LLM+human feedback. - Iterate and Optimize: Continuously refine metrics and evaluation flows to align with customer needs. If you’re working on LLM-powered applications, building high-quality Evals is one of the most impactful investments you can make. It’s not just about metrics — it’s about ensuring your app resonates with real-world users and delivers measurable value.

Using powerful LLMs (GPT-4) as an evaluator for smaller models is becoming the de facto standard. However, relying on closed-source models is suboptimal due to missing control, transparency, and versioning. 🤔 The recent paper "Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models" shows that open LLMs can match GPT-4 evaluation skills. 🚀 🔥𝗣𝗿𝗼𝗺𝗲𝘁𝗵𝗲𝘂𝘀 𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 1️⃣ Created a new dataset with 1000 scoring rubrics, 20K instructions (20 each), and 100K responses with feedback scores (1-5) generated by GPT-4 (5 each). → 100k training samples 2️⃣ Fine-tuned Llama-2-Chat-13B on this dataset (1️⃣) to generate the feedback (Prometheus 🔥) 3️⃣ Evaluated Prometheus on seen and unseen rubrics (including MT Bench), comparing correlation with human scores and GPT-4 scores ✨𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀 🥇 Scores a Pearson correlation of 0.897 with human evaluators, on par with GPT-4 (0.882), and outperforms GPT-3.5. (0.392) 🧑⚖️ Can be used a replacement of GPT-4 for LLM-as-a-Judge 🧬 High correlation with GPT-4 → due to imitation learning? 🔢 Requires 4 components in the input: prompt, generation to evaluate, a score rubric, and a reference generation. 😍 Prometheus can be further improved on training on customized rubrics and feedback, e.g. company specific domains 🧠 Can be used as a Reward Model for RLHF or for DPO to create preference pairs. 🤗 Dataset and Model available on Hugging Face Paper: https://lnkd.in/eXx-n_tx Dataset: https://lnkd.in/e8gVRGm4 Model: https://lnkd.in/eF9tKiTc Kudos to the researchers for this contribution to make AI more explainable, reproducible, and open! 🤗

Evaluating LLMs accurately/reliably is difficult, but we can usually automate the evaluation process with another (more powerful) LLM... Automatic metrics: Previously, generative text models were most commonly evaluated using automatic metrics like ROUGE and BLEU, which simply compare how well a model’s output matches a human-written target resopnse. In particular, BLEU score was commonly used to evaluatate machine translation models, while ROUGE was most often used for evaluating summarization models. Serious limitations: With modern LLMs, researchers began to notice that automatic metrics did a poor job of comprehensively capturing the quality of an LLM’s generations. Oftentimes, ROUGE scores were poorly correlated with human preferences—higher scores don’t seem to indicate a better generation/summary [1]. This problem is largely due to the open-ended nature of most tasks solved with LLMs. There can be many good responses to a prompt. LLM-as-a-judge [2] leverages a powerful LLM (e.g., GPT-4) to evaluate the quality of an LLM’s output. To evaluate an LLM with another LLM, there are three basic structures or strategies that we can employ: (1) Pairwise comparison: The LLM is shown a question with two responses and asked to choose the better response (or declare a tie). This approach was heavily utilized by models like Alpaca/Vicuna to evaluate model performance relative to proprietary LLMS like ChatGPT. (2) Single-answer grading: The LLM is shown a response with a single answer and asked to provide a score for the answer. This strategy is less reliable than pairwise comparison due to the need to assign an absolute score to the response. However, authors in [2] observe that GPT-4 can nonetheless assign relatively reliable/meaningful scores to responses. (3) Reference-guided grading: The LLM is provided a reference answer to the problem when being asked to grade a response. This strategy is useful for complex problems (e.g., reasoning or math) in which even GPT-4 may struggle with generating a correct answer. In these cases, having direct access to a correct response may aid the grading process. “LLM-as-a-judge offers two key benefits: scalability and explainability. It reduces the need for human involvement, enabling scalable benchmarks and fast iterations.” - from [2] Using MT-bench, authors in [2] evaluate the level of agreement between LLM-as-a-judge and humans (58 expert human annotators), where we see that there is a high level of agreement between these strategies. Such a finding caused this evaluation strategy to become incredibly popular for LLMs—it is currently the most widely-used and effective alternative to human evaluation. However, LLM-as-a-judge does suffer from notable limititations (e.g., position bias, verbosity bias, self-enhancement bias, etc.) that should be considered when interpretting data.

A proposed qualitative evaluation framework for Generative AI writing tools: This post is my first draft of an evaluation framework for assessing generative AI tools (e.g. Claude, ChatGPT, Gemini). It's something I’ve been working on with Ryan Low — originally in the interest of selecting the best option for Rotational. At some point we realized sharing these ideas might help us and others out there trying to pick the best AI solution for your company's writing needs. We want to be clear that this is not another LLM benchmarking tool. It's not about picking the solution that can count the r's in strawberry or repeatably do long division. This is more about the everyday human experience of using AI tools for our jobs, doing the kinds of things we do all day solving our customers' problems 🙂. We're trying to zoom in on things that directly impact our productivity, efficiency, and creativity. Do these resonate with anyone else out there? Has anyone else tried to do something like this? What other things would you add? Proposed Qualitative Evaluation Criteria 1 - Trust and Accuracy Do I trust it? How often does it say things that I know to be incorrect?  Do I feel safe? Do I understand how my data is being used when I interact with it? 2 - Autonomous Capabilities How much work will it do on my behalf? What kinds of research and summarization tasks will it do for me? Will it research candidates for me and draft targeted emails? Will it read documents from our corporate document drive and use the content to help us develop proposals? Will it review a technical paper, provided a URL? 3 - Context Management and Continuity How well does the tool maintain our conversation context? Not to sound silly, but does the tool remember me? Is it caching stuff? Is there a way for me to upload information about myself into the user interface so that I don’t have to continually reintroduce myself? Does it offer a way to group our conversations by project or my train of thought? Does it remember our past conversations? How far back? Can I get it to understand time from my perspective? 4 - User Experience Does the user interface feel intuitive? 5 - Images How does it do with images? Is it good at creating the kind of images that I need? Can the images it generates be used as-is or do they require modification? 6 - Integrations Does it integrate with our other tools (e.g. for project management, for video conferences, for storing documents, for sales, etc)? 7 - Trajectory Is it getting better? Does the tool seem to be improving based on community feedback? Am I getting better at using it?

🚀 Building a Financial Research News Bot with AI Agents & LLM-as-a-Judge Over the past few days, I’ve been experimenting with a multi-agent architecture designed to automate financial news research using LLMs as evaluators (aka “LLM-as-a-Judge”). The system’s goal: → Generate a concise and relevant financial news summary for a given region → Automatically evaluate whether each item is properly sourced with an external link and publication date → Iterate only when needed (e.g. if links are missing), to minimize cost and improve quality 🧠 Architecture Overview: I used the OpenAI Agents SDK. The system I created is made up of two specialized agents: 1. web_news_searcher – searches the web (via tools that query sources like Reuters) for the most recent impactful financial news, and produces a summary. 2. news_evaluator – reviews the summary and determines whether each news item is backed by a proper external source link and publication date. I used OpenAI’s built-in WebSearch tool to augment the web_news_searcher agent’s access to real-time information. These agents interact in an evaluation-refinement loop, stopping automatically when quality criteria are met.  The evaluator uses a simple feedback schema: → `successful` if all items are well sourced → `needs_links` if not, with detailed feedback injected into the next prompt 🧪 Use Case Tested: Various requests tested. This is an example of one of them: → Give me the latest 5 news items for the APAC region. Specify the source and date for each one. 📊 Results: - A complete and accurate 5-item summary - All items sourced with full links and publication dates - Evaluator scored it as `successful` — no refinement needed! 🎯 💡 Key Takeaways - Modular architecture with evaluator-optimizer loop helps manage quality without human intervention - Link completeness checks are essential to avoid hallucination in financial news - LLM-as-a-Judge patterns can guide multi-step, multi-agent reasoning with minimal prompt engineering - Cost-effective: runs only until feedback criteria are satisfied 🔗 Link to a public notebook or GitHub repo in the comment 👇 📅 Note that this method can easily scale to different regions or topics (e.g. ESG, tech, geopolitical)

LLM-as-a-Judge is a dangerous way to do evaluation. In most cases, it just doesn't work. While it gives a false impression of having a grasp on your system's performance, it lures you with general metrics such as correctness, faithfulness, or completeness. They hide several complexities: - What does "completeness" mean for your application? In the case of a marketing AI assistant, what characterizes a complete post from an incomplete one? If the score goes higher, does it mean that the post is better? - Often, these metrics are scores between 1-5. But what is the difference between a 3 and a 4? Does a 5 mean the output is perfect? What is perfect, then? - If you "calibrate" the LLM-as-a-judge with scores given by users during a test session, how do you ensure the LLM scoring matches user expectations? If I arbitrarily set all scores to 4, will I perform better than your model? However, if LLM-as-a-judge is limited, it doesn't mean it's impossible to evaluate your AI system. Here are some strategies you can adopt: - Online evaluation is the new king in the GenAI era Log and trace LLM outputs, retrieved chunks, routing… Each step of the process. Link it to user feedback as binary classification: was the final output good or bad? Then take a look at the data. Yourself, no LLM. Take notes and try to find patterns among good and bad traces. At this stage, you can use an LLM to help you find clusters within your notes, not the data. After taking this time, you'll already have some clues about how to improve the system. - Evaluate deterministic steps that come before the final output Was the retrieval system performant? Did the router correctly categorize the initial query? Those steps in the agentic system are deterministic, meaning you can evaluate them precisely: Retriever: Hit Rate@k, Mean Reciprocal Rank, Precision, Recall Router: Precision, Recall, F1-Score Create a small benchmark, synthetically or not, to evaluate offline those steps. It enables you to improve them later on individually (hybrid search instead of vector search, fine-tune a small classifier instead of relying on LLMs…) - Don't use tools that promise to externalize evaluation Your evaluation system is your moat. If you externalize it, not only will you have no control over your AI application, but you also won't be able to improve it. This is your problem, your users, your revenue. Evaluate your system. Not a generic one. All problems are different. Yours is unique as well. ... Those are some unequivocal ideas proposed by the AI community. Yet, I still see AI projects relying on LLM-as-a-judge and generic metrics among companies. Being able to evaluate your system gives you the power to improve it. So take the time to create the perfect evaluation for your use case.

Tackling Hallucination in LLMs: Mitigation & Evaluation Strategies As Large Language Models (LLMs) redefine how we interact with AI, one critical challenge is hallucination—when models generate false or misleading responses. This issue affects the reliability of LLMs, particularly in high-stakes applications like healthcare, legal, and education. To ensure trustworthiness, it’s essential to adopt robust strategies for mitigating and evaluating hallucination. The workflow outlined above presents a structured approach to addressing this challenge: 1️⃣ Hallucination QA Set Generation Starting with a raw corpus, we process knowledge bases and apply weighted sampling to create diverse, high-quality datasets. This includes generating baseline questions, multi-context queries, and complex reasoning tasks, ensuring a comprehensive evaluation framework. Rigorous filtering and quality checks ensure datasets are robust and aligned with real-world complexities. 2️⃣ Hallucination Benchmarking By pre-processing datasets, answers are categorized as correct or hallucinated, providing a benchmark for model performance. This phase involves tools like classification models and text generation to assess reliability under various conditions. 3️⃣ Hallucination Mitigation Strategies In-Context Learning: Enhancing output reliability by incorporating examples directly in the prompt. Retrieval-Augmented Generation: Supplementing model responses with real-time data retrieval. Parameter-Efficient Fine-Tuning: Fine-tuning targeted parts of the model for specific tasks. By implementing these strategies, we can significantly reduce hallucination risks, ensuring LLMs deliver accurate and context-aware responses across diverse applications. 💡 What strategies do you employ to minimize hallucination in AI systems? Let’s discuss and learn together in the comments!

LLM Evaluation: Why Testing AI Models Is No Longer Optional Organizations are deploying LLMs at an incredible pace—often treating them like high-performing employees who can execute tasks instantly. But here’s the uncomfortable question: Are we actually checking their work? Without rigorous evaluation, speed can easily mask hidden risks—hallucinations, bias, reasoning gaps, and unreliable outputs. LLM evaluation is essentially quality control for AI. It helps us: • Measure performance against ground truth • Identify blind spots and knowledge gaps • Detect bias and harmful outputs • Compare models using standardized benchmarks • Build trust with users and stakeholders In enterprise environments—especially regulated sectors like finance, healthcare, and public sector—evaluation isn’t just a best practice. It’s a governance requirement. Metrics like accuracy, recall, F1, coherence, latency, toxicity, BLEU, and ROUGE give us a multi-dimensional view of model behavior—not just “does it sound good?” Frameworks such as MMLU, HumanEval, TruthfulQA, GLUE, and IBM FM-Eval are becoming foundational to LLMOps and responsible AI programs. The real shift happening right now: AI is moving from experimentation → operational infrastructure And infrastructure must be measurable, auditable, and reliable. #AI #GenerativeAI #LLMOps #ResponsibleAI #AIGovernance #AIEngineering #EnterpriseAI #AgenticAI Image Credit : The Gen Academy

Here's the LLM evaluation stack I recommend to every team: Layer 1: Unit Tests (DeepEval) Stop treating AI as a mystery box. Integrate with Pytest to run assertions on every build. → Test individual components (retrievers, generators, tools) → Run in CI/CD to block regressions → Move from vibe-checking to deterministic engineering Layer 2: Metric Suite (50+ SOTA Metrics) Quantify performance with academic-grade metrics, not just "looks good" scores: → Hallucination: Is it making things up? → Faithfulness: Is it strictly grounded in your context? → Agentic Trajectory: Did it pick the right tool and use the correct arguments? → G-Eval: Define custom, subjective criteria in plain English. Layer 3: Synthetic Data Evolution Don't wait for user logs to find your bugs. → Generate thousands of "Golden" test cases from your docs in minutes → Automatically cover complex edge cases → Scale your testing without a single manual label Layer 4: Continuous Monitoring Evaluation doesn't stop at deployment. → Track performance drift in real-time → Get a "Rationale" (the why) for every production failure → A/B test prompt versions with statistical confidence DeepEval handles all 4 layers in one framework. One framework: ✓ 50+ research-backed metrics ✓ Pytest-native syntax ✓ Synthetic data generation ✓ Full Agent & RAG support This is how you ship AI with actual confidence. (100% Open-Source) GitHub Repo - https://lnkd.in/gQ3zCcZN Don't forget to ⭐️

I've spent countless hours building and evaluating AI systems. This is the 3-part evaluation roadmap I wish I had on day one. Evaluating an LLM system isn't one task. It's about measuring the performance of each component in the pipeline. You don't just test "the AI"; You test the retrieval, the generation, and the overall agentic workflow. 𝗣𝗮𝗿𝘁 𝟭: 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗻𝗴 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 (𝗧𝗵𝗲 𝗥𝗔𝗚 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲) Your system is only as good as the context it retrieves. 𝗞𝗲𝘆 𝗠𝗲𝘁𝗿𝗶𝗰𝘀: ↳ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗣𝗿𝗲𝗰𝗶𝘀𝗶𝗼𝗻: How much of the retrieved context is actually relevant vs. noise? ↳ 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗥𝗲𝗰𝗮𝗹𝗹: Did you retrieve all the necessary information to answer the query? ↳ 𝗡𝗗𝗖𝗚: How high up in the retrieved list are the most relevant documents? 𝗞𝗲𝘆 𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲𝘀: ↳ 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸: RAGAs Framework (Repo) https://lnkd.in/gAPdCRzh ↳ 𝗣𝗮𝗽𝗲𝗿: RAGAs Paper https://lnkd.in/gUKVe4ac 𝗣𝗮𝗿𝘁 𝟮: 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗻𝗴 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗧𝗵𝗲 𝗟𝗟𝗠'𝘀 𝗥𝗲𝘀𝗽𝗼𝗻𝘀𝗲) Once you have the context, how good is the model's actual output? 𝗞𝗲𝘆 𝗠𝗲𝘁𝗿𝗶𝗰𝘀: ↳ 𝗙𝗮𝗶𝘁𝗵𝗳𝘂𝗹𝗻𝗲𝘀𝘀: Does the answer stay grounded in the provided context, or does it start to hallucinate? ↳ 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲: Is the answer directly addressing the user's original prompt? ↳ 𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻 𝗙𝗼𝗹𝗹𝗼𝘄𝗶𝗻𝗴: Did the model adhere to the output format you requested? 𝗞𝗲𝘆 𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲𝘀: ↳ 𝗧𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲: LLM-as-Judge Paper https://lnkd.in/gyhaU5CC ↳ 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀: OpenAI Evals & LangChain Evals https://lnkd.in/g9rjmfGS https://lnkd.in/gmJt7ZBa 𝗣𝗮𝗿𝘁 𝟯: 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗻𝗴 𝘁𝗵𝗲 𝗔𝗴𝗲𝗻𝘁 (𝗧𝗵𝗲 𝗘𝗻𝗱-𝘁𝗼-𝗘𝗻𝗱 𝗦𝘆𝘀𝘁𝗲𝗺) Does the system actually accomplish the task from start to finish? 𝗞𝗲𝘆 𝗠𝗲𝘁𝗿𝗶𝗰𝘀: ↳ 𝗧𝗮𝘀𝗸 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗶𝗼𝗻 𝗥𝗮𝘁𝗲: Did the agent successfully achieve its final goal? This is your north star. ↳ 𝗧𝗼𝗼𝗹 𝗨𝘀𝗮𝗴𝗲 𝗔𝗰𝗰𝘂𝗿𝗮𝗰𝘆: Did it call the correct tools with the correct arguments? ↳ 𝗖𝗼𝘀𝘁/𝗟𝗮𝘁𝗲𝗻𝗰𝘆 𝗽𝗲𝗿 𝗧𝗮𝘀𝗸: How many tokens and how much time did it take to complete the task? 𝗞𝗲𝘆 𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲𝘀: ↳ 𝗚𝗼𝗼𝗴𝗹𝗲'𝘀 𝗔𝗗𝗞 𝗗𝗼𝗰𝘀: https://lnkd.in/g2TpCWsq ↳ 𝗗𝗲𝗲𝗽𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴(.)𝗔𝗜 𝗔𝗴𝗲𝗻𝘁𝘀 𝗘𝘃𝗮𝗹 𝗖𝗼𝘂𝗿𝘀𝗲: https://lnkd.in/gcY8WyjV Stop testing your AI like a monolith. Start evaluating the components like a systems engineer. That's how you build systems that you can actually trust. Save this roadmap. What's the hardest part of your current eval pipeline? ♻️ Repost this to help your network build better systems. ➕ Follow Shivani Virdi for more.