Evaluating LLM Performance in Complex Fact Linking

Unlocking the Next Era of RAG System Evaluation: Insights from the Latest Comprehensive Survey Retrieval-Augmented Generation (RAG) has become a cornerstone for enhancing large language models (LLMs), especially when accuracy, timeliness, and factual grounding are critical. However, as RAG systems grow in complexity-integrating dense retrieval, multi-source knowledge, and advanced reasoning-the challenge of evaluating their true effectiveness has intensified. A recent survey from leading academic and industrial research organizations delivers the most exhaustive analysis yet of RAG evaluation in the LLM era. Here are the key technical takeaways: 1. Multi-Scale Evaluation Frameworks The survey dissects RAG evaluation into internal and external dimensions. Internal evaluation targets the core components-retrieval and generation-assessing not just their standalone performance but also their interactions. External evaluation addresses system-wide factors like safety, robustness, and efficiency, which are increasingly vital as RAG systems are deployed in real-world, high-stakes environments. 2. Technical Anatomy of RAG Systems Under the hood, a typical RAG pipeline is split into two main sections: - Retrieval: Involves document chunking, embedding generation, and sophisticated retrieval strategies (sparse, dense, hybrid, or graph-based). Preprocessing such as corpus construction and intent recognition is essential for optimizing retrieval relevance and comprehensiveness. - Generation: The LLM synthesizes retrieved knowledge, leveraging advanced prompt engineering and reasoning techniques to produce contextually faithful responses. Post-processing may include entity recognition or translation, depending on the use case. 3. Diverse and Evolving Evaluation Metrics The survey catalogues a wide array of metrics: - Traditional IR Metrics: Precision@K, Recall@K, F1, MRR, NDCG, MAP for retrieval quality. - NLG Metrics: Exact Match, ROUGE, BLEU, METEOR, BertScore, and Coverage for generation accuracy and semantic fidelity. - LLM-Based Metrics: Recent trends show a rise in LLM-as-judge approaches (e.g., RAGAS, Databricks Eval), semantic perplexity, key point recall, FactScore, and representation-based methods like GPTScore and ARES. These enable nuanced, context-aware evaluation that better aligns with real-world user expectations. 4. Safety, Robustness, and Efficiency The survey highlights specialized benchmarks and metrics for: - Safety: Evaluating robustness to adversarial attacks (e.g., knowledge poisoning, retrieval hijacking), factual consistency, privacy leakage, and fairness. - Efficiency: Measuring latency (time to first token, total response time), resource utilization, and cost-effectiveness-crucial for scalable deployment.

It is easy to criticize LLM hallucinations but Google researchers just made a major leap toward solving them for statistical data. In the DataGemma paper (Sep ’24), they teach LLMs when to ask an external source instead of guessing. They propose two approaches: Retrieval interleaved generation (RIG) - the model injects natural language queries into its output, triggering fact retrieval from Data Commons. Retrieval augmented generation (RAG) - the model pulls full data tables into its context and reasons over them with a long-context LLM. The results are impressive: (1) RIG improved statistical accuracy from 5–17% to ~58% (2) RAG hit ~99% accuracy on direct citations (with some inference errors still remaining) (3) Users strongly preferred the new responses over baseline answers. As LLMs increasingly rely on external tools, teaching them "when to ask" may become as important as "how to answer." Paper https://lnkd.in/gaKY_VNE

My favorite paper from NeurIPS’24 shows us that frontier LLMs don’t pay very close attention to their context windows… Needle In A Haystack: The needle in a haystack test is the most common way to test LLMs with long context windows. The test is conducted via the following steps: 1. Place a fact / statement within a corpus of text. 2. Ask the LLM to generate the fact given the corpus as input. 3. Repeat this test while increasing the size of the corpus and placing the fact at different locations. From this test, we see if an LLM “pays attention” to different regions of a long context window, but this test purely examines whether the LLM is able to recall information from its context. Where does this fall short? Most tasks being solved by LLMs require more than information recall. The LLM may need to perform inference, manipulate knowledge, or reason in order to solve a task. With this in mind, we might wonder if we could generalize the needle in a haystack test to analyze more complex LLM capabilities under different context lengths. BABILong generalizes the needle in a haystack test to perform long context reasoning. The LLM is tested based upon its ability to reason over facts that are distributed in very long text corpora. Reasoning tasks that are tested include fact chaining, induction, deduction, counting, list / set comprehension, and more. Such reasoning tasks are challenging, especially when necessary information is scattered in a large context window. “Our evaluations show that popular LLMs effectively utilize only 10-20% of the context and their performance declines sharply with increased reasoning complexity.” - BABILong paper Can LLMs reason over long context? We see in the BABILong paper that most frontier LLMs struggle to solve long context reasoning problems. Even top LLMs like GPT-4 and Gemini-1.5 seem to consistently use only ~20% of their context window. In fact, most LLMS struggle to answer questions about facts in texts longer than 10,000 tokens! What can we do about this? First, we should just be aware of this finding! Be wary of using super long contexts, as they might deteriorate the LLM’s ability to solve more complex problems that require reasoning. However, we see in the BABILong paper that these issues can be mitigated with a few different approaches: - Using RAG is helpful. However, this approach only works up to a certain context length and has limitations (e.g., struggles to solve problems where the order of facts matters). - Recurrent transformers can answer questions about facts from very long contexts.

There is growing interest in using large language models (LLMs) to retrieve scientific literature and answer medical questions. However, the ability of LLMs to critically assess evidence and reason across multiple documents to provide recommendations at the same proficiency as domain experts remains poorly characterized. Systematic reviews (SRs), in which experts synthesize evidence across studies, are a cornerstone of clinical decision-making, research, and policy. Their rigorous evaluation of study quality and consistency makes them a strong source to evaluate expert reasoning, raising a simple question: Can LLMs match the conclusions of systematic reviews written by clinical experts when given access to the same studies? To explore this question, we present: 🎯 MedEvidence Benchmark: A human-curated benchmark of 284 questions (from 100 open-access SRs) across 10 medical specialties. All questions are manually transformed into closed-form question answering to facilitate evaluation. 📊 Large-scale evaluation on MedEvidence: We analyze 24 LLMs spanning general-domain, medical-finetuned, and reasoning models. Through our systematic evaluation, we find that: 1. Reasoning does not necessarily improve performance 2. Larger models do not consistently yield greater gains 3. Medical fine-tuning degrades accuracy on MedEvidence. Instead, most models show overconfidence, and, contrary to human experts, lack scientific skepticism toward low-quality findings. 😨 These results suggest that more work is still required before LLMs can reliably match the observations from expert-conducted SRs, even though these systems are already deployed and being used by clinicians! 📄Paper: https://lnkd.in/ghTa3pVA 🌐Website: https://lnkd.in/gvCTcsxR Huge shoutout to my incredible first co-authors, Christopher Polzak and Min Woo Sun, and to James Burgess, Yuhui Zhang, and Serena Yeung-Levy for their amazing contributions and collaboration.

MIT just published research on why ChatGPT struggles with state tracking. The problem isn't memory. It's how transformers encode position information. Current models use RoPE (rotary position encoding). It treats all words four positions apart the same way. Doesn't matter if it's "cat sat on box" or financial data changing over time. MIT-IBM Watson AI Lab built PaTH Attention to fix this. It outperforms RoPE on state tracking and sequential reasoning. Here's what this means for how you use LLMs today: 1. Audit where your LLM loses context in long documents Test with financial reports, legal contracts, or multi-step instructions. Track where the model misses state changes or sequential logic. Example: "Company X acquired Y in Q2, then sold Z in Q4" often gets confused. Current position encoding can't track entity relationships over time. 2. Break complex documents into state-aware chunks Don't feed 50-page contracts as single prompts. Segment by state changes: before acquisition, during transition, after close. Explicitly label each section's timeframe and context. This compensates for positional encoding limitations. 3. Use explicit state markers in your prompts Add "Current state:" before each major transition. Example: "Current state: Post-merger. Previous state: Pre-merger." Forces the model to treat position changes as data, not just distance. Reduces errors in multi-step reasoning by 40-60%. 4. Test LLM performance on conditional logic tasks Build test cases with "if-then" sequences over long contexts. Example: "If condition A occurs on page 5, apply rule B on page 20." Current models fail these because RoPE doesn't track causal relationships. Know your model's limits before deploying in production. 5. Prioritize reasoning over retrieval for complex documents RAG (retrieval-augmented generation) won't fix state tracking issues. It retrieves chunks but doesn't understand how states evolve. For contracts, regulations, or multi-step workflows, use specialized parsing. Position encoding is the bottleneck, not retrieval accuracy. 6. Watch for next-gen models with adaptive position encoding PaTH Attention is research, not yet in production models. But it signals where LLM architecture is heading. Models that track state changes will replace current transformers. Plan your document processing stack accordingly. Why this matters: You're using LLMs on tasks they structurally can't handle well. Financial analysis, legal review, code debugging over long contexts. All require state tracking that RoPE fundamentally doesn't provide. MIT just showed the problem and the solution. Most teams won't adjust their workflows until new models ship. You can compensate for these limitations now. Found this helpful? Follow Arturo Ferreira.