What is the agent tool use design pattern? - Microsoft Copilot Studio Tutorial

(futuristic music) - What is the tool use design pattern, how does it help AI agents do more, and what should we plan for when using it to build AI agents? We're going to answer these questions in the fourth lesson of the AI Agents for Beginners course. In this course, we take you from concept to code, covering the fundamentals of building AI agent, and in this short video follows along with the written lesson, including translations and the code samples that you can find at the link above and below the video. So let's get started looking at what the tool use design pattern is. And while LLMs are very effective on their own in generating content, tasks that are requested by users sometimes require external tools to complete, and the tool design pattern allows the LLMs to interact with these tools, such as a calculators, APIs to tell us about flight status, or a function within our application handling currency exchange. To understand the benefits of this, let's take a look at a few use cases. For example, an AI agent can create database queries using tools for generating code and be able to retrieve real-time information from a customer database of the customer's travel loyalty points and travel preferences. Or an AI agent can be connected to a CRM system and to answer specific questions about a customer's travel booking rather than needing a human intervention to do so. And while we've been talking about single use tools, agents can also combine tools in order to automate workflows. For example, analyzing an email, retrieving the relevant knowledge-based information, and the forwarding it, that email, to customer service representative to make that process more efficient. There's a lot to consider when talking about tool calling, for example, security, making sure AI agents have only the type of access required, and handling errors because, sadly, we don't live in a perfect world where the services that the agent might be using are always running. Maybe one day. But we will cover these topics more in depth in our trustworthy AI agents lesson and AI agents in production. But let's head over to our code and build a tool calling AI agent. Okay, so now we're at our code editor, and we're using the Semantic Kernel tool example with a notebook. Again, you can find this code at the link above and below this video. And we've been covering tool calling in the other lessons as well, but I want to show you just a few other kind of options and abilities that we have within this kind of world of tool calling. And in this case we have this DestinationsPlugin. We're going be using Semantic Kernel, and we have one that just has a list of destinations that are available, basically a list of destinations, and then the other one is their actual availability. So you can kind of think of this maybe as two different databases or maybe one that has, you know, a list of things that are destinations available and then the current status, whether they're, you know, being able to be booked or not. So we will also show this in terms of the function behavior. So this is another kind of setting book built into Semantic Kernel, where we can have this either be auto, so it basically allows the agent to choose when to call a function or when not to, whenever it deems it appropriate, or required when we want to make sure that the agent is calling a function, depending on the kind of use case and the scenario that we have. Again, we're setting up the agent like before, but in this case we want to have three user inputs, and I want to show kind of the value of working with tool calling here. First one is the user input. The question is, you know, what destinations are available? Then it's going to ask about specific destination, and then it's going to ask, are there any destinations available not in Europe? And we can look at the interaction here to see how this goes. So first is, what destinations are available? And as expected, this is going to call the get_destinations plugin, our function, and this result is going to give back our list and then it can go ahead and show that list to the user. And this, again, is just saying this is just getting the destinations, right? Not just specifically about the availability of them, about what ones can I actually book. And then it's going to actually, the user's going to ask about, is Barcelona available? So again, kind of similarly query, but in this case, instead of calling the get_destinations, since it's given us a specific destination, we can call the get_availability and see that the result is that Barcelona is unavailable, and the travel agent responds accordingly, saying Barcelona is currently unavailable. But I want to show this as as another kind of example of why, you know, we're not just making API recalls here but also using the power of large language models to interpret natural language, because in this case, you know, "Are there any vacation destinations available not in Europe?" we don't have to as a developers to building these agents define which ones are in Europe or not because large language model has all of that within its sort of base training data. So in this case, you don't need to have a new function saying is in Europe or not in Europe or anything like that, right? We can just, first that large language model is going to call the get_availability. In this case, it already has deemed that Tokyo, Japan is not in Europe and that New York is not also in Europe, but it's going to call that specific function to get availability to see which ones are not available, and then we get the result for both of them, and then perfectly says that New York is available. It also gives an kind of extra information that Tokyo, Japan is currently unavailable. So again, this really shows you the power of using agents, large language models, to both use their abilities to interpret natural language, have that sort of interaction, as well as tool calling to give it the specific data based on that interpretation and that plan from the agent itself. So that was tool calling. We're going to continue to use this as a feature in other samples, but we'll see in the next lesson. (crescendoing music)