From the course: Build with AI: SQL AI Agents in Production
Join today to access over 25,500 courses taught by industry experts.
Building and integrating a testing framework
From the course: Build with AI: SQL AI Agents in Production
Building and integrating a testing framework
So far, we've explored the core components of a SQL AI agent, from prompt templates to error handling. One of the final steps before deploying an agent to production, it's testing its functionality and performance. In this lesson, we'll work through an example of testing framework that focuses on two key scenarios. End-to-end evaluation of the agent's ability to answer user questions and evaluation of the debugger agent's ability to fix errors during runtime. Although our focus here is module evaluation, the same principles apply to other parts of the system, such as prompt handling and safety validation. You can use this approach as a foundation and extend it to cover the agent's core components. For this demo, we'll use the 0404B Jupyter Notebook in the Chapter 4 folder. We will evaluate seven OpenAI models covering a wide range of capabilities. This includes the model we have used throughout the course, GPT-4.0, as well as more advanced models available at the time of recording…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.