How to Use Knowledge Graphs in Llms

Agentic systems don't just benefit from Small Language Models. They architecturally require them, paired with knowledge graphs. Here's the technical reality most teams miss. 🎯 The Workload Mismatch Agents execute 60-80% repetitive tasks: intent classification, parameter extraction, tool coordination. These need <100ms latency at millions of daily requests. Physics doesn't negotiate. Model size determines speed. But agents still need complex reasoning capability. 🧠 The Graph Solution The breakthrough: separate knowledge storage from reasoning capability. LLMs store facts in parameters. Inefficient. Graph-augmented SLMs externalize knowledge to structured triples (entity-relationship-entity), use 3-7B parameters purely for reasoning. Knowledge Graph of Thoughts: Same SLM solves 2x more tasks when querying graphs vs. processing raw text. Cost drops from $187 to $5 per task. Multi-hop reasoning becomes graph traversal, not token generation. Token consumption drops 18-30%. Hallucination reduces through fact grounding. 💰 The Economics At 1B requests/year: GPT-5 approach: $190K+ 7B SLM + graph infrastructure: $1.5-19K One production system: $13M annual savings, 80%→94% coverage by caching knowledge as graph operations. ⚡ The Threshold Below 3B parameters: Models can't formulate effective graph queries Above 3B: Models excel at coordinating retrieval and synthesis over structured knowledge Modern 7B models (Qwen2.5, DeepSeek-R1-Distill, Phi-3) now outperform 30-70B models from 2023 on graph-based reasoning benchmarks. 🏗️ The Correct Architecture Production agents converge on this pattern: Query → Classifier SLM → Graph construction/update → Specialist SLMs query graph → Multi-hop traversal → Response synthesis → (5% escalate to LLM) The graph provides: External memory across reasoning steps Fact grounding to prevent hallucination Reasoning scaffold for complex inference 🔐 Why This Matters Edge deployment: 5GB graph + 7B model runs locally on laptops Privacy: Medical/financial data never leaves premises Latency: Graph queries are deterministic <50ms operations Updates: Modify graph triples without model retraining Real case: Clinical diagnostic agent on physician laptop. Patient symptoms → graph traversal → diagnosis in 80ms. Zero external transmission. 🎓 The Separation of Concerns Graphs handle: relationship queries, continuous updates, auditability SLMs handle: query formulation, reasoning coordination, synthesis LLMs conflate both functions in one monolith. This drives their size and cost. Agent tasks follow this pattern: understand intent → retrieve structured knowledge → reason over relationships → execute action → update knowledge state. Graphs make each step explicit. SLMs provide coordination intelligence. Together, they outperform larger models on unstructured data at 10-36x lower cost. Are you still processing agent tasks with 70B+ models on raw text, or have you separated knowledge (graphs) from reasoning (SLMs)?

I've been working with knowledge graphs and semantic web technologies since 2012. RDF. OWL. SPARQL. Jena. Protege. Over a decade of building ontologies, designing knowledge graphs, and writing SPARQL queries. And one frustration has never gone away: the gap between what knowledge graphs can do and what people are willing to put up with to use them. The technology is powerful. Ontological reasoning, automatic inference, structured knowledge representation — these are genuinely valuable capabilities. But the barrier to entry has always killed adoption before the value becomes visible. LLMs finally change that equation. So I built KeplAI (v0.1, needs improvement), an open-source platform that puts an AI layer on top of the standards-based semantic web stack (Apache Jena, Fuseki, RDF TripleStore, SPARQL), Qdrant. What it does: → Ask questions in plain English, get SPARQL + results → Extract structured knowledge from text automatically → Import and manage multiple ontologies with conflict detection → Entity disambiguation via vector embeddings → OWL reasoning built in — inference happens at the data layer → Full web UI for non-technical users The key insight: don't dumb down the semantic web. Keep the ontological rigor underneath. Put a natural-language interface on top. LLMs are great at understanding language but hallucinate facts. Knowledge graphs are great at storing facts but are hard to query naturally. Together, they're complementary in a way that feels almost too clean. I wrote a deep dive on the blog covering: - Why ontologies matter more than people think - The multi-ontology problem (and how to handle it) - Why LLM-to-SPARQL is harder than LLM-to-SQL - The self-correcting NLQ pipeline I built to handle predicate hallucination - Where knowledge graphs + LLMs are heading Full post: https://lnkd.in/ectxdNx3 GitHub: https://lnkd.in/ew5zSvNZ

Recently was studying knowledge graphs, I came across this excellent paper showing how the LinkedIn ML team implemented KGs for customer service. Customer service is extremely important, as TAT matters, and is one of the most common LLM usecases. The Approach: Merging RAG with KGs The essence of LinkedIn's method lies in constructing a Knowledge Graph from historical issue tickets, preserving both the intra-issue structure and the inter-issue relations. This graph-based structure allows for a more nuanced understanding and retrieval of past issues, ensuring that the system can accurately pinpoint the most relevant past instances when responding to new customer queries. Key Components of the Method: Knowledge Graph Construction - Intra-Issue Structure: Each customer service ticket is parsed into a tree-like structure, where sections of the ticket (such as the issue summary, description, and solution) are treated as nodes connected by hierarchical relationships. This retains the logical flow and relationships within a single ticket. - Inter-Issue Connections: The method then connects these trees into a comprehensive graph, linking tickets based on explicit references (e.g., one ticket being related to another) and implicit connections derived from semantic similarities. This interconnected graph ensures that related issues are easily accessible. Query Parsing and Retrieval: - Entity Recognition and Intent Detection: When a customer submits a query, the system identifies the key entities (e.g., "login issue") and the intent (e.g., "how to resolve"). This allows the system to navigate the Knowledge Graph effectively. - Sub-Graph Retrieval: The system then uses embedding-based retrieval to locate the most relevant sub-graphs within the Knowledge Graph. By preserving the context and relationships within these sub-graphs, the system can generate more accurate and comprehensive answers. Answer Generation: Leveraging the retrieved sub-graphs, the LLM generates a response that is both accurate and contextually relevant, drawing on the full breadth of related past issues rather than just isolated text segments. Empirical Results and Impact The effectiveness of this approach was empirically validated through rigorous testing against traditional text-based retrieval methods. The results were impressive, with a 77.6% improvement in Mean Reciprocal Rank (MRR) and a 0.32 increase in BLEU scores, underscoring the enhanced accuracy and quality of answers generated by the system. Perhaps more compelling is the real-world impact observed after deploying this system within LinkedIn's customer service team. The integration of RAG with KGs led to a 28.6% reduction in the median time required to resolve customer issues.

What’s the point of a massive context window if using over 5% of it causes the model to melt down? Bigger windows are great for demos. They crumble in production. When we stuff prompts with pages of maybe-relevant text and hope for the best, we pay in three ways: 1️⃣ Quality: attention gets diluted, and the model hedges, contradicts, or hallucinates. 2️⃣ Latency & cost: every extra token slows you down, and costs rise rapidly. 3️⃣ Governance: no provenance, no trust, no way to debug and resolve issues. A better approach is a knowledge graph + GraphRAG pipeline that feeds the model the most relevant data with context instead of all the things it might need with no top-level organization. ✅ How it works at a high level: Model your world: extract entities (people, products, accounts, APIs) and typed relationships (owns, depends on, complies with) from docs, code, tickets, CRM, and wikis. GraphRAG retrieval: traverse the graph to pull a minimal subgraph with facts, paths, and citations, directly tied to the question. Compact context, rich signal: summarize those nodes and edges with provenance, then prompt. The model reasons over structure instead of slogging through sludge. Closed loop: capture new facts from interactions and update the graph so the system gets sharper over time. ✅ A 30-day path to validate it for your use cases: Week 1: define a lightweight ontology for 10–15 core entities/relations built around a high-value workflow. Week 2: build extractors (rules + LLMs) and load into a graph store. Week 3: wire GraphRAG (graph traversal → summarization → prompt). Week 4: run head-to-head tasks against your current RAG; compare accuracy, tokens, latency, and provenance coverage. Large context windows drive cool headlines and demos. Knowledge graphs + GraphRAG work in production, even for customer-facing use cases.

Exciting breakthrough in Retrieval Augmented Generation (RAG): Researchers have developed GFM-RAG, the first Graph Foundation Model for enhancing LLM knowledge retrieval. >> Key Innovations Novel Architecture: GFM-RAG introduces a query-dependent Graph Neural Network that can process complex knowledge relationships in a single step, dramatically improving both efficiency and accuracy compared to traditional multi-step approaches. Under the Hood - Constructs a knowledge graph index from documents to capture relationships between information - Uses a 6-layer query-dependent message passing neural network with 512-dimensional hidden states - Implements DistMult message functions and sum aggregation for graph processing - Pre-trains on 60 knowledge graphs containing over 14M triples and 700k documents >> Performance Highlights The system achieves state-of-the-art results across multiple datasets, outperforming existing methods by significant margins: - Up to 19.8% improvement in retrieval accuracy - 10x faster processing compared to multi-step approaches - Demonstrates strong zero-shot generalization across 7 different domains >> Impact This breakthrough by researchers from Monash University, Nanjing University of Science and Technology, and Griffith University represents a significant step forward in making LLMs more knowledgeable and efficient. The system's ability to scale and transfer across domains makes it particularly valuable for real-world applications.

Your RAG system just failed another complex query. The issue isn't your embeddings or vector database - it's that traditional RAG treats every document chunk like it exists in a vacuum. Graph RAG solves this. Let’s break it down: 𝗡𝗮𝗶𝘃𝗲 𝗥𝗔𝗚 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲: 1. User asks a question 2. Query gets converted to vector embeddings 3. System retrieves most similar chunks 4. LLM generates answer from those chunks This works great for simple Q&A, but here's where it breaks down: • Each chunk exists in isolation - no understanding of relationships between entities • Struggles with questions requiring synthesis across multiple documents • Can't connect the dots when information is scattered across your knowledge base • Limited to one external knowledge source 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚 𝗣𝗶𝗽𝗲𝗹𝗶𝗻𝗲: 1. Extract entities and relationships from documents using LLMs 2. Build a knowledge graph storing these connections 3. When queried, traverse the graph to find related information 4. Retrieve both relevant chunks AND their relationship context 5. Generate comprehensive answers using the enriched context Instead of just finding similar text, Graph RAG can follow entity relationships to discover relevant information that might not be textually similar to your query. 𝗪𝗵𝗲𝗻 𝘁𝗼 𝘂𝘀𝗲 𝘄𝗵𝗶𝗰𝗵: ✅ 𝗡𝗮𝗶𝘃𝗲 𝗥𝗔𝗚 for: • Simple fact-finding questions • Well-defined document sections • When speed is critical • Smaller, focused knowledge bases ✅ 𝗚𝗿𝗮𝗽𝗵 𝗥𝗔𝗚 for: • Complex reasoning across multiple documents • Questions about relationships between entities • Summarization tasks requiring synthesis • Large, interconnected knowledge bases 𝗜𝗻 𝘀𝗵𝗼𝗿𝘁: Traditional RAG treats documents as isolated chunks, limiting complex queries. Graph RAG builds knowledge graphs to understand entity relationships, enabling sophisticated reasoning across entire knowledge bases. 📕 Notebook Implementing GraphRAG with Neo4j: https://lnkd.in/dubFBKMg ✍️ Blog: https://lnkd.in/dkxbDxbH

Framework for Question Answering Over Custom Knowledge Graphs BYOKG-RAG addresses entity linking errors and generalization issues in knowledge graph QA systems by separating LLM and graph retrieval responsibilities. 🎯 Approach: LLMs generate graph artifacts (question entities, candidate answers, reasoning paths, OpenCypher queries). Specialized graph tools perform entity linking and context retrieval. The system iteratively refines linking and retrieval before answer generation. 📊 Results: 4.5% improvement over second-best method Tested on 5 benchmarks with diverse KG types Demonstrates improved generalization to custom KGs Framework is open-source 🔗 Read the paper: Amazon Science: https://lnkd.in/g98HWVRt Joint Optimization of Graph Retrieval and Language Model Reasoning This paper trains a graph retriever end-to-end with an LLM, eliminating the separation between retrieval and reasoning in existing graph RAG systems. 🌟 Technical Features: Attention-based mechanism for multi-hop entity navigation with noise filtering Encodes structural knowledge via soft tokens and verbalized graphs Uses LLM logits as implicit feedback to optimize retriever Operates without predefined ground-truth entities 📈 Results: State-of-the-art performance on 3 QA benchmarks Effective in open-domain settings without entity annotations Demonstrates scalability for large-scale graph expansion 🔗 Read the paper: Amazon Science: https://lnkd.in/gTbUZ9UV

What if we added symbolic logic and multi-agent planning to GraphRAG, for retrieving and reasoning over chemistry knowledge? That’s MOSES. GraphRAG pairs LLMs with knowledge graphs (KG) to improve retrieval and grounding through entities and relationships. But in scientific domains, where relationships are multiscale, logic-heavy, and often implicit, LLMs need additional scaffolding. Two mechanisms help: 🔹Ontology as a compass: formalize hierarchy, properties and constraints to enable precise querying and logical inference. 🔹Multi-agent systems as workers: divide tasks into preprocessing, planning, validation, and refinement steps. MOSES (Multi-agent Ontology System for Explainable Knowledge Synthesis) combines both for chemistry.  Its ontology-based, agentic workflow: 1️⃣ Generates hypothesis: proposes likely mechanisms, relevant entities, and the structure of a complete answer. 2️⃣ Parses query: extracts intent and key entities, maps them to ontology classes, and classifies the query type. 3️⃣ Plans and executes: formulates a detailed execution plan and retrieve structured information from the KG. 4️⃣ Validates & iterates: Checks and refines results for completeness and logical coherence; format final output. This is less about producing "better answers" than explainable answers, with a visible trail of what was asked, what was found, how concepts were connected, and where evidence originates. Based on consistent feedback from scientists, this traceability and explainability are what build trust, and make LLMs a credible partner in scientific discovery. 📄 MOSES: combining automated ontology construction with a multi-agent system for explainable chemical knowledge reasoning, ChemRxiv, October 1, 2025 🔗 https://lnkd.in/e89vB6_V

A hidden layer of insight is waiting in your data  Here’s what most RAG setups miss  ☑ They fetch facts fast   ☑ They never see how those facts link together  That gap is why plain RAG stalls when you need the why. Enter GraphRAG and its next step, Agentic GraphRAG.  Instead of loose chunks, they stitch a graph:  ☑ Nodes are the entities you care about   ☑ Edges are the relationships between them  Your LLM now walks the graph and reasons over connections. A quick security incident example  It’s 3 AM. An alarm reports a breach.   Fifty CVE IDs pour in.  Which CVE is critical? Which is noise?  Agentic GraphRAG jumps in:  ☑ Pulls the incident data into the graph   ☑ Runs reasoning across known software, versions, past exploits   ☑ Ranks the threats and writes a short human readable note   ☑ Suggests next steps like a senior analyst would  The result feels like an autonomous analyst that understands relationships, reasons dynamically and even self corrects. If you want a RAG that does more than fetch, try building a knowledge graph first.  I’m Shrey Shah & I share daily guides on AI.  If this helped, hit the ♻️ reshare button to help someone else level up their AI.