Multiturn conversations

The code that we've written out so far simulates a very simple exchange with our model. And we can kind of visualize this conversation inside of a chat box like this. So we sent in a request asking in something like what's one plus one? We got a very simple response back, something like the answer is two. Naturally, we might want to continue this conversation at some point in time. So we might want to send in a follow-up asking something like and three more. And then we would expect to get back a response saying something like adding three to two would result in five. To have a multi-message conversation like this, there's something really critical you need to understand around the Bedrock API and Cloud itself. That is that Bedrock and Cloud do not store any messages. None of the messages you send to it get stored in any way, and none of the responses you get back are stored in any way. So if you ever want to have some conversation going on, where you have multiple messages that maintain a context or a flow, then there are two things you need to do. You need to manually inside of your code, maintain a list of all the messages that you are exchanging. And second, you need to make sure that you provide that entire list of messages with every follow-up request that you make. So let's go into some detail on this entire idea just so it's really clear and you understand what's going on here. The first thing I want to do is run just a little bit of sample code. So I've still got the code we wrote in the last video up here where we asked what's 1 plus 1, and we got back the 1 plus 1 is 2. Now I'm going to paste in some additional code snippet here that I wrote ahead of time. So in this additional code snippet that I just pasted in, I'm making another call to Bedrock, and this time I'm providing a message of just and three more. So this simulates that screenshot I showed you a moment ago. Now if I run this cell, I will eventually get back a response that doesn't make any sense whatsoever. So let me show you a diagram to make sure it's clear why we got this very strange text. So on the left-hand side, I've got the messages that I'm including in my request off to Bedrock. I start off with just a message of what's one plus one. We send that in and we get back an assistant message of one plus one is two, exactly what we expect. When I then executed the code that you saw just a moment ago, that extra little code snippet, I sent in a request that contained just one single message that said and three more. Now, when we send this and three more in, Cloud is gonna do its best to give you a response. In our case, the response we actually got back didn't make a lot of sense, but it's doing the best job it possibly can because all it knows is the fact that we said and three more. So to solve this problem, here's what we're going to do. As I mentioned just a moment ago, we're going to manly maintain a list of these messages and make sure we provide that list with every single follow-up request. So as a demonstration of what we're gonna do going forward, whenever we want to have a multi-turn conversation, we're still gonna send in this original user message. We're gonna get back the assistant response, and then we're going to include that assistant message in our list of messages. So we're gonna kind of move it on over there to the left-hand side. Then whenever we want to continue this conversation, we're going to append onto the end of the list another user message where we ask and three more. And now when we take this full list of messages and make our request, Cloud will have the full context of the conversation. It will see all three messages and we'll know exactly what we are talking about when we say and three more. So we'll get back hopefully a response of something like three plus two is five. Let's go back over to our notebook and try to simulate a multi-turn conversation like this. Okay, so back over here, I'm gonna delete that little code snippet I pasted in very quickly. Then inside this earlier code cell, I'm gonna write out three helper functions. These helper functions are just gonna make it a little bit easier for us to maintain the entire context of this conversation and maintain that list of messages. So right above user message, I'm going to put in a function definition of add user message. I'm going to assume that I'm going to call this function with a list of messages and some text. I'm then going to indent userMessage like so. I'm going to replace what's one plus one right here with the text argument that we're gonna pass in. And then right after we define userMessage, I'm gonna add that into the list of messages. Next up, I'm going to copy this function entirely. I'm gonna paste it right down here. I will rename it to add assistant message. I will change the name of this variable to be assistant message, change the role to be assistant, and then finally update messages.append right here to be assistant message instead. So now we have two helper functions that can very easily make a message type for us, either a user or an assistant message, and append it into an existing list of messages. Then one more helper function. Down here, I'm going to make it just a little bit easier for us to call our model, get a response, and then get the actual text we care about out of it. So I'm going to also define a function right here called chat. It will take in a list of messages. I'm going to indent response. I'm gonna pass in the list of messages like so. And I'm going to return from this function, response, output, message. And do you remember everything you have to put in there? It's then content, that's right. We then get a zero and text like so. So now with these three functions, we can very easily simulate a full conversation. Let me show you how we would do it. Down here in this cell, I'm gonna paste in a couple of comments that I wrote ahead of time just to guide myself and make sure I'm doing this entire conversation correctly. So here's my list of comments. We're gonna put in one line of code for each comment that I've added in. But don't worry, each line of code is gonna be very short and simple. First, I'm gonna make a starting list of messages. So initially I start off with no communication whatsoever. I'm then going to add in the initial user question. So that initial user message of what's one plus one. To do so, I'll call add user message, pass in the list of messages, and then the second argument is gonna be the actual text I want inside this message. So in this case, it'll be what's one plus one. And then immediately after that, I'm just gonna print out messages really quickly to make sure that we are on track. So I'm gonna run this and it looks like, yep, we added in our user message correctly. So I've got the role of user, content is a list and a Python dictionary inside there. So I'm gonna continue on to the next step. I'm gonna pass the list of messages into that chat function we made to get back an answer. So when we call that chat function, it's gonna reach out to Bedrock and call our API. So we'll say answer is chat and pass in the list of messages. And once again, just to make sure I'm on track, I'll print out answer. I'm gonna run that. And after a short pause, I see, yep, we got it correctly. We got our answer of one plus one equals two. So now we have this answer back as a plain string. We need to take that and add it as an assistant message into our list. And again, we're gonna do so by using that helper function we just made. So we'll say add assistant message, pass in our list with the answer we just got back. And once again, print out messages just to make sure we're doing this correctly. So now we will see, yes, we are in fact building up our conversation. We're building up our list of messages. We've got first user and then assistant. We've got what's one plus one and then the answer we got back. So now we can follow up with the user's additional question, which was add user message. And we'll say, and three more added to that. And we'll skip checking messages again. We'll just go ahead and get our follow-up answer by calling our API again with a full list of messages. I'll print out the final answer, and let's see what we end up with. And so sure enough, starting with the result of that, and we add three more, we end up getting five. So now it's very clear that we are maintaining the context of this conversation by including all the previous messages inside of our followup request. Now, this might initially seem a little bit tedious and even this code right here might seem a little bit confusing. I don't blame you if it is confusing. Don't worry, this is the kind of thing that you're gonna get really used to very quickly. And there's just one last thing I want to mention around lists of messages, like the one we have assembled right here. And that is whenever we send in a list of messages to the API, we always need to make sure that we are alternating roles between each message. So in other words, we should always have a user and then an assistant and then a user and then an assistant. We should never have two user messages in a row and we should also never have two assistant messages in a row or any variation thereof. So we always wanna make sure that we are varying or alternating those roles.