Solving LLM Onboarding and Developer Tooling Challenges

If you’re building with LLMs, these are 10 toolkits I highly recommend getting familiar with 👇 Whether you’re an engineer, researcher, PM, or infra lead, these tools are shaping how GenAI systems get built, debugged, fine-tuned, and scaled today. They form the core of production-grade AI, across RAG, agents, multimodal, evaluation, and more. → AI-Native IDEs (Cursor, JetBrains Junie, Copilot X) Modern IDEs now embed LLMs to accelerate coding, testing, and debugging. They go beyond autocomplete, understanding repo structure, generating unit tests, and optimizing workflows. → Multi-Agent Frameworks (CrewAI, AutoGen, LangGraph) Useful when one model isn’t enough. These frameworks let you build role-based agents (e.g. planner, retriever, coder) that collaborate and coordinate across complex tasks. → Inference Engines (Fireworks AI, vLLM, TGI) Designed for high-throughput, low-latency LLM serving. They handle open models, fine-tuned variants, and multimodal inputs, essential for scaling to production. → Data Frameworks for RAG (LlamaIndex, Haystack, RAGflow) Builds the bridge between your data and the LLM. These frameworks handle parsing, chunking, retrieval, and indexing to ground model outputs in enterprise knowledge. → Vector Databases (Pinecone, Weaviate, Qdrant, Chroma) Backbone of semantic search. They store embeddings and power retrieval in RAG, recommendations, and memory systems using fast nearest-neighbor algorithms. → Evaluation & Benchmarking (Fireworks AI Eval Protocol, Ragas, TruLens) Lets you test for accuracy, hallucinations, regressions, and preference alignment. Core to validating model behavior across prompts, versions, or fine-tuning runs. → Memory Systems (MEM-0, LangChain Memory, Milvus Hybrid) Enables agents to retain past interactions. Useful for building persistent assistants, session-aware tools, and long-term personalized workflows. → Agent Observability (LangSmith, HoneyHive, Arize AI Phoenix) Debugging LLM chains is non-trivial. These tools surface traces, logs, and step-by-step reasoning so you can inspect and iterate with confidence. → Fine-Tuning & Reward Stacks (PEFT, LoRA, Fireworks AI RLHF/RLVR) Supports adapting base models efficiently or aligning behavior using reward models. Great for domain tuning, personalization, and safety alignment. → Multimodal Toolkits (CLIP, BLIP-2, Florence-2, GPT-4o APIs) Text is just one modality. These toolkits let you build agents that understand images, audio, and video, enabling richer input/output capabilities. If you're deep in AI infra or systems, print this out, build a test project around each, and experiment with how they fit together. You’ll learn more in a weekend with these tools than from hours of reading docs. What’s one tool you’d add to this list? 👇 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI infrastructure insights, and subscribe to my newsletter for deeper technical breakdowns: 🔗 https://lnkd.in/dpBNr6Jg

The key to fully leveraging LLMs for work, in my experience: 1. Cursor - AI-first integration makes this a perfect "vibe coding" platform IMO. 2. Managing context window - Todo lists, consistently updated documentation, making sure the LLM has the context it needs to not have to make assumptions that lead to failure or codebase cruft. 90% of issues that crop up are the result of the LLM making an unfounded assumption based on incomplete context. If they have the full context, the success rates are extremely high for any given code change. 3. Test Driven development - Have the LLM write tests for pretty much everything it does, and test Fractally at all levels of abstraction. Your codebase should be 1/2 tests or more IMO. Its the best way to incrementally build a large project without it getting insanely complex and ultimately unmanageable for LLMs to get right. 4. MCP integrations - Superpowers for your LLM. Google Chrome Web dev console and other similar integrations have been a game changer for me, allowing for number 5: 5. Automate the above by forcing the LLMs into loops, either in chat or by having them write custom self editing review scripts. For instance I often prompt them "There are significant whitespace/positioning issues, use Google's Dev Console MCP and Cursor's browser tool" (can do screenshots in the loop without any setup/issues!), This allows a closed iterative loop on fixing front end design issues. I can have it iterate this loop as many times as needed until it completes the job fully. MCPs for managing most other external systems allowing for you to remove yourself from time consuming and annoying debug loops. 6. Multiple tabs/agent-teams working together - Because you can have multiple tabs/agents open at one time in Cursor, you can create massive efficiency gains if you plan it properly. For instance, have a main orchestrator agent managing a primary markdown Todo list that is split up between 3-4 teams. The primary orchestrator creates a massive todo list for these 3-4 teams, but done in a non-overlapping way. Then open up new tabs for each of these teams, prompt them to learn the codebase fully so they are up to speed, and let them do that, Then set them in a loop working on the team todo list. You can create a massive project that actually works, extremely quickly if you can manage the process end to end with planning from the start and by putting these puzzle pieces together correctly and managing your context well.

𝐈 𝐡𝐚𝐯𝐞 𝐬𝐩𝐞𝐧𝐭 𝐭𝐡𝐞 𝐥𝐚𝐬𝐭 𝐲𝐞𝐚𝐫 𝐡𝐞𝐥𝐩𝐢𝐧𝐠 𝐄𝐧𝐭𝐞𝐫𝐩𝐫𝐢𝐬𝐞𝐬 𝐦𝐨𝐯𝐞 𝐟𝐫𝐨𝐦 "𝐈𝐌𝐏𝐑𝐄𝐒𝐒𝐈𝐕𝐄 𝐃𝐄𝐌𝐎𝐒" 𝐭𝐨 "𝐑𝐄𝐋𝐈𝐀𝐁𝐋𝐄 𝐀𝐈 𝐀𝐆𝐄𝐍𝐓𝐒".  The pattern is always the same:  Teams nail the LLM integration and think the hard part is done, then realize they have built 20% of what production actually requires. 𝐇𝐞𝐫𝐞 𝐢𝐬 𝐰𝐡𝐲 𝐞𝐚𝐜𝐡 𝐛𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐛𝐥𝐨𝐜𝐤 𝐦𝐚𝐭𝐭𝐞𝐫𝐬: Reasoning Engine (LLM): Just the Beginning • Interprets intent and generates responses • Without surrounding infrastructure, it is just expensive autocomplete • Real engineering starts when you ask: "How does this agent make decisions it can defend?" Context Assembly: Your Competitive Moat • Where RAG, memory stores, and knowledge retrieval converge • Identical LLMs produce vastly different results based purely on context quality • Prompt engineering does not matter if you are feeding the model irrelevant information Planning Layer: What to Do Next • Breaks goals into steps and decides actions before acting • Separates thinking from doing • Poor planning = agents that thrash or make circular progress Guardrails & Policy Engine: Non-Negotiable • Defines what APIs the agent can call, what data it can access • Determines which decisions require human approval • One misconfigured tool call can cascade into serious business impact Memory Store: Enables Continuity • Short-term state + long-term memory across interactions • Without it, every conversation starts from zero • Context window isn't memory it's just scratchpad Validation & Feedback Loop: How Agents Improve • Logging isn't learning • Capture user corrections, edge cases, quality signals • Best teams treat every interaction as potential training data Observability: Makes the Invisible Visible • When your agent fails, can you trace exactly why? • Which context was retrieved? What reasoning path? What was the token cost? • If you can not answer in under 60 seconds, debugging will kill velocity Cost & Performance Controls: POC vs Product • Intelligent model routing, caching, token optimization are not premature they are survival • Monthly bills can drop 70% with zero accuracy loss through smarter routing What most teams miss: They build top-down (UI → LLM → tools)  when they should build bottom-up (infrastructure → observability → guardrails → reasoning). These 11 building blocks are not theoretical. They are what every production agent eventually requires either through intentional design or painful iteration. 𝐖𝐡𝐢𝐜𝐡 𝐛𝐥𝐨𝐜𝐤 𝐚𝐫𝐞 𝐲𝐨𝐮 𝐜𝐮𝐫𝐫𝐞𝐧𝐭𝐥𝐲 𝐮𝐧𝐝𝐞𝐫𝐢𝐧𝐯𝐞𝐬𝐭𝐢𝐧𝐠 𝐢𝐧? ♻️ Repost this to help your network get started ➕ Follow Anurag(Anu) Karuparti for more PS: If you found this valuable, join my weekly newsletter where I document the real-world journey of AI transformation. ✉️ Free subscription: https://lnkd.in/exc4upeq #GenAI #AIAgents

"Function calling isn’t working." "My Search tool is broken." "The agent isn't doing what I expect with BigQuery." Sound familiar? When a tool fails in an AI agent, the instinct is often to blame the framework 😁 And while we love (!) the feedback, as I get into the weeds with customers, we often find the issue hiding somewhere else. So it becomes important to start seeing the agent and its tools as a layer cake and apply classic software engineering discipline: isolate the failure by debugging layer by layer. Here’s the 4-layer framework for debugging tool-use with agents, and how to use adk web to do it: 1️⃣ The Tool Layer: Does your tool's code work in isolation? Before you even look at a trace, run your function with a hardcoded input. If it fails here, it's a bug in your tool's logic. 2️⃣ The Model Layer: Is the LLM generating the correct intent? This is where traces are invaluable. In adk web, look at the trace for the step right before the tool call. You can see the exact prompt sent to the model and the raw LLM output. Is the model choosing the right tool? Are the parameters plausible? If not, the issue is your prompt or tool description. 3️⃣ The Connection Layer: This is where the model's request meets your code. Is there a mismatch? Use adk web to check the exact arguments the LLM tried to pass to your function. Are the parameter names correct? Is a number being passed as a string? The trace makes it obvious if the LLM's understanding doesn't match your function's signature. 4️⃣ The Framework Layer: If the first three layers look good, now we look at the orchestration. How did the agent handle the tool's output? Use adk web to check the full trace is the story of your agent's execution. You can see the data returned by the tool and the subsequent LLM call where the agent decides what to do next. This is where you'll spot issues in your agent's logic flow. This methodical approach, powered by observability tools like traces, turns a vague "my agent is broken" into a more precise diagnosis. How do you debug your agents tool-use? Comment below if a deep dive into any of these area would be useful! #AI #Agents #Gemini #DeveloperTools #FunctionCalling #Debugging #Observability

Toolchaining sounds simple: give an LLM a bunch of tools and let it figure it out. In practice? It breaks. Devang S Ram Mohan and our research team at Scale AI found that LLMs struggle when they both plan and orchestrate tool calls — context gets bloated, outputs get corrupted, and accuracy drops. The fix: separate planning from execution. 👉 The LLM generates a plan (as Python or JSON). 👉 A dedicated executor + expert utilities handle the messy data. The result: near-human accuracy, faster turnaround, and even novel solutions the humans hadn’t thought of. We’ve now integrated this “Plan as Python” approach into our MLE toolkit and it’s already running in production. Read the full write-up here: https://lnkd.in/g4KPZNni

🚀 𝗛𝗼𝘄 𝗜 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗲𝗱 𝗟𝗟𝗠𝘀 𝗶𝗻𝘁𝗼 𝗦𝗽𝗿𝗶𝗻𝗴 𝗕𝗼𝗼𝘁 𝘁𝗼 𝗠𝗮𝗸𝗲 𝗗𝗲𝗯𝘂𝗴𝗴𝗶𝗻𝗴 & 𝗧𝗲𝘀𝘁𝗶𝗻𝗴 𝗔𝗹𝗺𝗼𝘀𝘁 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰 Last week, I was chasing a memory leak 🤯 — and it hit me. I was stuck doing the same old backend grind: ✅ Parsing logs manually ✅ Writing repetitive unit tests ✅ Updating Swagger docs by hand Then I remembered the GPT integration we’d built for our internal tools. Within minutes, it: 🧠 Explained the root cause 🧪 Generated full test scenarios ⚡ Suggested performance optimizations And that’s when it clicked: LLMs aren’t replacing backend developers. They amplify us.   💡𝐖𝐡𝐲 𝐁𝐚𝐜𝐤𝐞𝐧𝐝 𝐓𝐞𝐚𝐦𝐬 𝐒𝐭𝐢𝐥𝐥 𝐋𝐚𝐠 𝐁𝐞𝐡𝐢𝐧𝐝 While frontend teams are shipping AI-powered features, backend developers are buried in: 🔍 Unit testing 📝 API documentation 🐛 Log analysis 🧠 Googling "Spring Boot best practices" for the 100th time These aren’t tech challenges — they’re productivity bottlenecks. The solution? Not another framework. It’s architecting AI into your workflow.   🧠 𝐓𝐡𝐞 𝐒𝐦𝐚𝐫𝐭 𝐋𝐋𝐌 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐅𝐥𝐨𝐰: 1️⃣ Request Sanitization → Remove sensitive data before sending to LLM 2️⃣ Context Building → Include Spring Boot-specific patterns and domain knowledge 3️⃣ LLM Processing → GPT-4 / Claude / Llama does the reasoning 4️⃣ Response Validation → Enforce internal coding standards 5️⃣ Integration → Feed insights back into your dev workflow 𝐑𝐞𝐚𝐥-𝐖𝐨𝐫𝐥𝐝 𝐈𝐦𝐩𝐚𝐜𝐭: 💥 Debugging time → 30 mins → 8 mins 💥 Test coverage → 65% → 85% (auto-generated edge cases) 💥 Documentation → Always up-to-date 💥 Developer velocity → +40% faster feature delivery Engineers stopped repeating tasks and started solving actual business problems. 🔐 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 & 𝐑𝐞𝐥𝐢𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐅𝐢𝐫𝐬𝐭 Before integrating any LLM: ✅ Sanitize sensitive data (PII, tokens, configs) ✅ Isolate network calls ✅ Enable audit logging ✅ Design fallback strategy for LLM downtime Additional wins: 🔄 Provider flexibility → Switch between GPT-4, Claude, Llama seamlessly ⚡ Performance → Async, caching, circuit breakers 📊 Observability → Track usage, latency, and cost   💬 Let’s Talk What’s your biggest backend productivity pain right now? 👉 Writing unit tests? 👉 Debugging production incidents? 👉 Keeping docs up-to-date? Drop your thoughts below — I’d love to discuss AI-powered backend productivity. #SpringBoot #Java #AI #LLM #BackendDevelopment #GPT4 #Claude3 #Llama3 #Microservices #SystemDesign #WalmartGlobalTech #CodeAutomation #TestAutomation #DeveloperTools #SoftwareEngineering #EngineeringExcellence #ArtificialIntelligence

AI coding LLMs and tools are improving rapidly. There is a massive amount of value and velocity teams can unlock by using them correctly. One reminder I recently shared internally at Productboard that’s worth repeating more broadly👇 It’s critical to start with a strong product specification. Spend the first 1–2 hours iterating on the spec definition to ensure all requirements are clear and there are no surprises mid-implementation. A few practical tips on how to do that: 🔹 Paste (or even better, pull via MCP) the specs you got from your PM into a Markdown file 🔹 Ask Claude: “Ask me any questions needed to make sure you deeply understand the feature we will be building.” You might get 40–60 questions back - ideally use something like WhisperFlow so you don’t spend the next two hours just answering them 🔹 Ask Claude: “Propose three very different approaches to building this feature and explain their pros and cons in terms of complexity, maintainability, and user value.” Then iterate toward the approach that makes the most sense 🔹 Ask Claude: “Research the codebase, put together an implementation plan for this feature, and come back with additional product questions that need to be answered before implementation.” Context engineering is just as critical. A few tips there: 🔹 Use a “Research → Plan → Implement” staged flow, fully wiping the context window between each stage instead of relying on automatic compaction 🔹 Spend significant time reading, reviewing, and adjusting the outputs of each stage 🔹 Use research sub-agents heavily - you may need to explicitly prompt for this depending on the tool and LLM you’re using When it comes to implementation quality: 🔹 Make sure you truly understand every line of code you push into a PR 🔹 Having the agent walk you through the changes and explain non-obvious parts (especially around libraries or frameworks) is often a great idea Tooling matters more than ever: 🔹 Make sure you deeply understand the features and tricks of the coding tools you use - not easy when tools like Claude Code and Cursor ship updates almost daily 🔹 Invest in AI tooling configuration in your repos 🔹 Invest in better linters - the best teams are often doubling the number of linter rules compared to pre-AI days, giving agents fast and precise feedback 🔹 Constantly update your AGENTS.md / Claude.md files as you notice behaviors that should be adjusted - top teams update these almost daily And finally: 🔹 Share your tips and tricks with colleagues How are you and your teams approaching AI-assisted coding today? What practices have made the biggest difference for you so far?

Most teams start with building AI evals. After a cohort with 700+ AI PMs and engineers, I understood why that breaks down. LLMs introduce 4 unique challenges: - Non-determinism, no single "right answer" - Subjective nature of “quality” - The scarcity of labeled data for specific failure modes - Black-box model updates from foundation model providers The instinct is to jump straight to building evaluators. But without error analysis first, you're measuring the wrong things. You need ~100 high-quality traces before writing a single eval. Analyze them. Let failure modes emerge bottom-up. Only then do your metrics mean something. So how do we deal with this at scale? During the cohort by Hamel H. and Shreya Shankar one idea stood out to me: the Continuous Improvement Flywheel. Here’s how it works: (If you're just starting with evals, check out the free "Mastering AI Evals: A Complete Guide" linked at the bottom) 𝟏. 𝐂𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧 (𝐂𝐈) 𝐟𝐨𝐫 𝐋𝐋𝐌𝐬 - CI protects against regressions on known issues - The core artifact is your Golden Dataset - Automated evaluators are your regression tests 𝟐. 𝐂𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐃𝐞𝐥𝐢𝐯𝐞𝐫𝐲 (𝐂𝐃) 𝐚𝐧𝐝 𝐎𝐧𝐥𝐢𝐧𝐞 𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠 - Focus on observability in production - You need full LLM traces: inputs, outputs, tool calls, retrieval, errors, etc. - Example tools: LangSmith, Arize, W&B Weave 𝟑. 𝐑𝐮𝐧𝐧𝐢𝐧𝐠 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐞𝐝 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐨𝐫𝐬 𝐢𝐧 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 - Run async evals on a small % of traffic (1–5%) - Use sync evaluators as guardrails that take action: Reject (block the request), Retry (re-run the LLM pipeline), Fallback (switch to an alternative) 𝟒. 𝐓𝐡𝐞 𝐂𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐈𝐦𝐩𝐫𝐨𝐯𝐞𝐦𝐞𝐧𝐭 𝐅𝐥𝐲𝐰𝐡𝐞𝐞𝐥 AI evaluation is not a one-time task. It’s a feedback loop: a. Develop & Analyze b. Measure & Build Evals c. CI Setup d. Deploy (CD) with Observability e. Monitor Online Performance f. Identify Drift, New Failures, or Product Issues g. Re-Analyze (Error Analysis) h. Update Evaluation Artifacts i. Improve Pipeline j. Re-deploy & Iterate Details in the infographic. 𝟓. 𝐅𝐫𝐞𝐞 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞𝐬 𝐭𝐨 𝐆𝐨 𝐃𝐞𝐞𝐩𝐞𝐫 🔗 Mastering AI Evals: a Complete Guide: https://lnkd.in/dHUFrkVs 🔗 Error Analysis Process: https://lnkd.in/d9tZcvVT 🔗 AI Evals: A Massive FAQ (Google Drive): https://lnkd.in/eUJbuQFG 🔗 My infographic as a PDF after subscribing here: https://lnkd.in/dkBsZ-ZY Hope that helps! --- P.S. Interested in AI Evals? I recommend the cohort I mentioned. 3,000+ students have gone through it so far. Next session: January 26. A 25% discount you won't find elsewhere: https://lnkd.in/eU5PbYzw

𝐌𝐨𝐬𝐭 𝐋𝐋𝐌 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐃𝐞𝐩𝐥𝐨𝐲𝐦𝐞𝐧𝐭𝐬 𝐟𝐚𝐢𝐥 𝐧𝐨𝐭 𝐛𝐞𝐜𝐚𝐮𝐬𝐞 𝐨𝐟 𝐌𝐨𝐝𝐞𝐥𝐬, But because teams ignore these 6 Critical CI/CD Patterns. 𝐓𝐡𝐞 𝟔 𝐄𝐬𝐬𝐞𝐧𝐭𝐢𝐚𝐥 𝐏𝐚𝐭𝐭𝐞𝐫𝐧𝐬 𝐟𝐨𝐫 𝐋𝐋𝐌 𝐂𝐈/𝐂𝐃: 𝟏. 𝐌𝐎𝐍𝐎𝐋𝐈𝐓𝐇𝐈𝐂 𝐁𝐔𝐈𝐋𝐃𝐒 Problem: Single large codebase, slow rebuilds Solution: Modularized components—prompts, models, APIs separate 𝟐. 𝐋𝐀𝐂𝐊 𝐎𝐅 𝐀𝐔𝐓𝐎𝐌𝐀𝐓𝐄𝐃 𝐓𝐄𝐒𝐓𝐈𝐍𝐆 Problem: Manual testing, bugs in production Solution: Automated tests for prompts, responses, performance, quality 𝟑. 𝐈𝐍𝐒𝐔𝐅𝐅𝐈𝐂𝐈𝐄𝐍𝐓 𝐄𝐍𝐕𝐈𝐑𝐎𝐍𝐌𝐄𝐍𝐓 𝐏𝐀𝐑𝐈𝐓𝐘 Problem: Dev/Testing/Prod differ, "works on my machine" Solution: Identical configs, containerization, IaC, same model versions 𝟒. 𝐏𝐎𝐎𝐑 𝐕𝐄𝐑𝐒𝐈𝐎𝐍 𝐂𝐎𝐍𝐓𝐑𝐎𝐋 Problem: Unclear commits, no traceability Solution: Clear branching, descriptive messages, code reviews, tag releases 𝟓. 𝐎𝐕𝐄𝐑𝐂𝐎𝐌𝐏𝐋𝐈𝐂𝐀𝐓𝐄𝐃 𝐏𝐈𝐏𝐄𝐋𝐈𝐍𝐄𝐒 Problem: Too many stages, slow feedback Solution: Streamlined stages—Build → Test → Deploy, parallel execution 𝟔. 𝐈𝐍𝐀𝐃𝐄𝐐𝐔𝐀𝐓𝐄 𝐒𝐄𝐂𝐔𝐑𝐈𝐓𝐘 Problem: Secrets in commits, exposed keys Solution: Secrets management, branch protection, credential scanning LLM-Specific Considerations: - Version prompts like code - Pin model versions—avoid "latest" - Monitor costs in pipeline - Automate output quality gates - Test for prompt injection - A/B test infrastructure My Recommendations: DO: Modularize, automate testing, version everything, simplify pipelines, protect secrets DON'T: Deploy untested, use different environments, skip reviews, overcomplicate, commit keys Truth: LLM apps need MORE CI/CD discipline. Without proper pipelines, you're debugging at 3 AM. Which mistake cost you most time? ♻️ Repost to help your network ➕ Follow Jaswindder for more #DevOps #MLOps #LLM #GenAI

LLMs are changing the way I think about software. For most of my career, I've thought deterministically. Decision trees, test coverage for every branch, predictable paths through code. If the user asks X, fetch Y and so on. This works when you control the inputs. But conversational interfaces break this model. When I built an AI Assistant for my book tracking app QuietReads (see comments), I couldn't predict which context any question would need. "What should I read next?" requires the want-to-read list. "Something like my last book, but shorter" requires recently finished books and filtering by length. "What were my thoughts on that dystopian novel?" requires notes. The traditional solution is routing logic that guesses what data to fetch. It gets unwieldy fast. Tool calling offers an alternative: define well-described functions and let the model decide which to call. You're moving routing decisions from compile-time (hardcoded logic) to runtime (model inference). The model becomes the router. With good tool schemas and capable models, this behavior is predictable, and you can bound costs with max tool call limits. The tool descriptions matter as much as the code. A phrase like "Always call this BEFORE recommending books" changes model behavior significantly. Tool schemas are a new API surface, with all the versioning and maintenance implications that come with it. There are significant tradeoffs. Costs vary based on conversation complexity. Testing becomes more complex because you're evaluating model behavior, not just code paths. But you gain flexibility and a clean separation between what data exists and when to fetch it. Tool calling is really a form of context engineering. You are trying to figure out what to give the LLM and navigate tradeoffs around token costs, accuracy, and performance. For an old backend developer like me, building AI-powered applications has required a change in mindset. More details, code examples, and a discussion on my tradeoffs can be found on my blog: https://lnkd.in/eskB9hf5