Troubleshooting Guides for Software Products

"Function calling isn’t working." "My Search tool is broken." "The agent isn't doing what I expect with BigQuery." Sound familiar? When a tool fails in an AI agent, the instinct is often to blame the framework 😁 And while we love (!) the feedback, as I get into the weeds with customers, we often find the issue hiding somewhere else. So it becomes important to start seeing the agent and its tools as a layer cake and apply classic software engineering discipline: isolate the failure by debugging layer by layer. Here’s the 4-layer framework for debugging tool-use with agents, and how to use adk web to do it: 1️⃣ The Tool Layer: Does your tool's code work in isolation? Before you even look at a trace, run your function with a hardcoded input. If it fails here, it's a bug in your tool's logic. 2️⃣ The Model Layer: Is the LLM generating the correct intent? This is where traces are invaluable. In adk web, look at the trace for the step right before the tool call. You can see the exact prompt sent to the model and the raw LLM output. Is the model choosing the right tool? Are the parameters plausible? If not, the issue is your prompt or tool description. 3️⃣ The Connection Layer: This is where the model's request meets your code. Is there a mismatch? Use adk web to check the exact arguments the LLM tried to pass to your function. Are the parameter names correct? Is a number being passed as a string? The trace makes it obvious if the LLM's understanding doesn't match your function's signature. 4️⃣ The Framework Layer: If the first three layers look good, now we look at the orchestration. How did the agent handle the tool's output? Use adk web to check the full trace is the story of your agent's execution. You can see the data returned by the tool and the subsequent LLM call where the agent decides what to do next. This is where you'll spot issues in your agent's logic flow. This methodical approach, powered by observability tools like traces, turns a vague "my agent is broken" into a more precise diagnosis. How do you debug your agents tool-use? Comment below if a deep dive into any of these area would be useful! #AI #Agents #Gemini #DeveloperTools #FunctionCalling #Debugging #Observability

I've spent over 12 years in DevOps and cloud. Here’s a summary of 10 brutal troubleshooting facts I’ve learned: 1) Check logs first, always – Logs contain the first clues; learn how to filter, search, and analyze them efficiently. 2) Trace the request flow – Understand how a request moves through the system to pinpoint failures faster. 3) Use process of elimination – Isolate components one by one to find the root cause instead of guessing. 4) Know the difference between infra and app issues – Is it a misconfigured server, network problem, or bad code? 5) Validate external dependencies – If your service relies on APIs, databases, or third-party tools, check their status. 6) Check system resource limits – Running out of memory, CPU, or disk can cause random failures. 7) Reproduce the issue in a test environment – If possible, recreate the failure to understand it better. 8) Keep a "known issues" doc – If something breaks often, document the fix so you (or others) don’t waste time. 9) Use health checks effectively – Proper liveness and readiness probes can detect and prevent hidden failures. 10) Know when to escalate – If you've checked the usual suspects and still can't fix it, don't waste time, get help. 40K+ read my free weekday daily TechOps Examples newsletter: https://lnkd.in/gg3RQsRK What do we cover: DevOps, Cloud, Kubernetes, IaC, GitOps, MLOps 🔁 Consider a Repost if this is helpful

Your ETL Job Broke? No Problem. 💡Reproduce the Problem - Use a test environment to recreate the issue on a similar dataset. Isolating the most time-consuming part of the pipeline may help identify delays during data aggregation. 💡Narrow Down Causes - Log files are your friend. Reviewing these may help find overlapping queries created by multiple data fetches. - Review your code. Nested loops could be increasing execution time. - Identify single-threaded operations. Parallelizing processes could decrease total time of execution. 💡Collaborate for Solutions - Gather stakeholders together, and troubleshoot. The faster everyone comes together to troubleshoot reduces total down time which translates into money saved and maintained SLAs. 💡Implement the Fix - Reduce query execution times by optimizing joins and indexing - Use Python Pandas instead of nested loops to manipulate data faster. ➡️A Quick Review - Tools like AWS CloudWatch, New Relic, and Dynatrace are invaluable when it comes to tracking application performance and resource bottlenecks trends over time to find the root cause of the problem. - Examine the system in sections in order to isolate issues. Check the database, infrastructure, code, etc. to get a detailed view of operations. - Make incremental changes. Test. Repeat. Iteration is key to narrowing down the list of potential causes.