Sign in to continue learning
Functions, Tools and Agents with LangChain
Intermediate
1h44m
Topics
AI Frameworks
Agents
Chatbots
Generative Models
Prompt Engineering
RAG
a new capability added to the OpenAI API a few months ago. We'll go over how exactly to use this, and some tips and tricks for getting best results. OpenAI has fine-tuned some of their most recent models to accept additional parameters for function calling. These models are fine-tuned to determine if it's relevant to call one of these functions, In this lesson, we'll use the OpenAI SDK directly, To walk through this example we're going to imagine that we have a function that we think is interesting to provide to the language model. And we'll go over what interesting means later on, because there are a bunch of different use cases for this new parameter. Here, we're going to define a getCurrentWeather function. This is an example from OpenAI themselves, when they first released this functionality. This is a good example to use because getting the current weather is something that the language model can't necessarily do by itself. And so, we often want to connect language models to functions In this example, we hard code the information that's returned, but in production this could be hitting a weather API or some external source of knowledge. OpenAI has exposed a new parameter called functions through which you can pass a list of function definitions. The full function definition for the above is right here. As you can see it's a list and then the element in the list there's only one because we're only passing one function and here it is this JSON object with a few different parameters. You've got a name paramete,r and this is the name of the function. You've then got a description parameter, And then, next you have this parameter object. And in here, there's properties. Properties is itself another object, and we can see Each of these elements has a type, string, and then, a description. Unit is an enum, because we want it to be either Celsius or Fahrenheit. And so we can reflect that here. We can also convey that the only required parameter is location, we're going to pass these functions directly to the language model, the description parameter here. And then, And so, the language model will use these descriptions Any information you want the language model to have in order to determine whether to call a function or how to call a function should be in the description here or here. Let's then import the OpenAI SDK and call the chat completion endpoint. First, we're going to specify the model. We're going to make sure to specify one of the more recent ones that has this capability. Next, we're going to pass in our messages defined above. Let's run this and see what we get. Let's look at the full response. We can see that the message we get back has role of assistant, has null for content, and instead has this function call parameter which has two objects, name and arguments. Name is get current weather. This is the same name as the function that we passed in, and then arguments is this JSON blob. Let's take a closer look at the response message. Again, content is now empty. And function call is this dictionary. The arguments parameter in function call is a JSON dictionary itself, so we can use JSON.loads to load this into a Python dictionary. The arguments that it passed back can be directly passed into OpenAI doesn't directly call the function. We still have to do that ourselves. Rather, it just tells us what function to call, that's the name, and what the arguments to that function should be. It's also worth noting that although this is trained to return a JSON blob, it's actually not strictly enforced. What happens if the message that you pass in isn't related to the function at all? Let's try it out and see what happens then. Let's create a new set of messages, this time ones that can see that the message that's returned has content as normal, and it doesn't have that function call parameter. What's going on under the hood is that the model is determining whether to use a function or not. There are additional parameters that we can pass in to force the model to use or not to use a function. Let's take a look at those. That additional parameter is the function call parameter. By default, it's set to auto. This means that the language model chooses. This is what we've been doing so far. There's two other modes that we can use it in. In the first mode, we can force it to call a function. This is good if we always want to return the function, and we'll see some use cases for that later on in the lesson. As you can see, because we're using auto and we're letting the language model choose what to do, here it recognizes that it doesn't need to call the function and so it's responding as before with role and content only. Another mode for function call that we can use is none. This forces the language model not to use any of the functions provided. In this example it doesn't really matter because the content high doesn't need the function call anyways. But what happens when the messages should call function here? It should call the get current weather function. But when we look down, we still see the usual role and content. That's because we're forcing it not to call the function. And so, The final option for the function call parameter is forcing it to call a function. Let's see how to do that. And if we look at the response we can see that in fact we do get this function call object returned and it's got name get current weather with some arguments. Just for fun let's see what happens if we pass in a message that doesn't need to call the function but we force it to. So here, it's making up San Francisco, California. If we run it again, it keeps on calling San Francisco, California. Some final things to note. First, the functions themselves and the descriptions count against the token usage limit that you pass to OpenAI. If we comment out functions and function call, we can see that prompt tokens goes down to 15. In this particular example, the function definition and the function description are taking up a lot of the tokens. This is important to note because OpenAI models have a token limit on them. And so, as you're constructing your messages to pass to OpenAI, Finally, let's now take a look at how you can pass some of these function calls and the results of actually doing the function calls back into the language model. This is important because oftentimes you want to use the language model to determine what function to call, then run that function, but then pass it back into the language model to get a final response. We'll then take this message and we'll append it to our list of messages. We can then simulate calling the getCurrentWeather function with the arguments that the language model provides. Let's save this to a variable and then we can append a new message to the list representing the response from the function that we just call. We do this with a new type of message. Notice that it has a role equal to function. This is used to convey to the language model that it's the response of calling a function. We then also pass in the name, which is the name of the function, and a content variable, which we can set up equal to observation, which we calculated above. If we then call the language model with this list of messages, we can see that the language model takes the response of the observation and converts it into a nice natural language response. The current weather in Boston is 72 degrees Fahrenheit with a sunny and windy forecast. That's all for this lesson where we introduced OpenAI function calling and how to use it with the OpenAI SDK directly. In the next lessons, we're gonna cover how to combine this with langchain primitives to make it easier and faster to use this functionality.
