You'll now build your tool calling agent using the OpenAI SDK. And then wrap it in the MLflow ResponsesAgent interface. Okay, let's code. Now that we have finished lab one and set our initial governance, we're going to go through our Lab 2, our agent.py, and then our Lab 3 deployment notebook so we can actually deploy the agent with the service principle's credentials. So I'm going to go back just one step, and I'm going to make sure that my service principles, so I'm going to go to my folder that contains all my data. I'm going to share permissions because I want to make sure that my service principle can actually run. Just going to make sure that that is added. Now let's go into our lab two agent.py. So this is our orchestration. So we talked about using the MLflow ResponsesAgent interface. So first we need to install key capabilities. So we use the Databricks SDK, we use databricks_openai, and we pull in our UCFunctionToolkit. If we wanted to use a retriever tool as well, so a vector search, if we were building a RAG chatbot, we have the VectorSearchRetrieverTool as an option. We of course need to import OpenAI. And then we need to select our LLM. So if we select a natively available Databricks model that has tool calling enabled, we can use the meta-llama-3.3-70b-instruct model. We can have our SYSTEM_PROMPT. So you are an HR Data Scientist and Analytics expert. You have access to HR analytics tools that provide insights into workforce performance, retention, and operational metrics. So a little bit more about the role, some feedback for when you're answering questions, and then the tools you have available. So this is your analyze_performance and analyze_operations tools that we've built and registered in Unity Catalog. And then you need to define your tools for your agent. So we're going to scroll down and so you can see the tools that we have already defined and registered in Unity Catalog. So you can use these user-defined functions in Unity Catalog as agent tools. If you wanted to use a vector database, you can easily enable that as well. And if I go all the way to the bottom, you can see that your ToolCallingAgent has your LLM endpoint, so the endpoint that we've defined. So this is our Meta Llama 3 70B model. our tools, so the tools that are going to allow the LLM access. to data, and that's going to bind our ToolCallingAgent. Now, if you already have an existing agent, all you have to do is wrap your existing agent with the MLflow ResponsesAgent interface, which we've talked about in our last lecture. If you're curious on how to wrap your existing agent with the MLflow ResponsesAgent interface, This is actually wrapped, so I'll show you what that looks like. So here we can see the class representing a tool calling agent, and we can see here that it implements the ResponsesAgent interface. So when you have a class that implements the interface ResponsesAgent, inside your class there should be two methods. And these are the predict and predict_stream methods, which wrap the output of the existing agent. And if you're more curious on how this works or the step-by-step process, or if you're looking to build a LangChain agent and wrap it, you can refer to our documentation links. Now I'm going to show you exactly how to produce this agent.py notebook. If I go over to Playground, I can choose a model for tool calling. If you see this skill enabled, this little emoji of a tool, that means it's a tool calling agent that's enabled. Now, these are just natively available. in Databricks, you can bring in an external model if you have anything custom, you can bring that in as well. So if you want to use your own OpenAI model, feel free to do so, and then you just need your key in order to use it. Here I'm going to use my Meta Llama 3.1 70B Instruct model. And then for my tools, I'm going to add the tools that I've created as UC functions. So I can go to my clientcare.hr_data.analyze_operations. Save that tool. Okay, add analyze performance. And I can select all my tools that are available in HR data. I'm just going to grab the named ones so you can see exactly how our agent is interacting with them. We can also add in a Vector Search endpoints or use MCP Servers. You can add in a system prompt. As you saw from our agent.py file, we have a very complex system prompt. So I'm just going to put in a little bit so you can see where that ends up. And so I can start asking questions to test this agent before I get that instrumentation code. So tell me about performance Here we can see that using our Meta Llama 3.3 70B instruct model. So we can see that the LLM pulled up our analyze_performance function, pulled up information around performance, so the operations we gave it. But now it also has a full explanation involved. We can see trace capabilities to see if it ended up using more than one tool or the total latency. I can also ask about operations. So to gain insights into the operational metrics of different departments, I'm going to use the function analyze operations. And then it's going to give me this thought process of what this data means, what I might find interesting. So a full analysis on the data it has access to. If I think this looks good, I can go and create the agent notebook. This will take me to my driver notebook which explains that we are authoring a tool-calling MLflow's ResponsesAgent that uses the OpenAI client. Now, just because this notebook and our example use the OpenAI SDK, our agent framework is compatible with any agent authoring framework, so you can use LlamaIndex, LangChain, straight Python in order to build these agents. But going down to defining and writing your agent.py file. Here we can see all the capabilities that we went through on the agent.py. We have the LLM endpoint, we have the system prompt. We have the tools that are available. If we wanted to, we can add in a vector search database. And then of course, we have our class ToolCallingAgent. And then when we log the model to MLflow, we have the MLflow logging of the model. And of course, we have our LLM endpoint. So the LLM that's going to be used for our agent as the brain, and then the tools that are going to give that LLM access to our data for our tool calling agent. So again, this is the exact same information that's available on our agent.py file. So if you use the cell magic at the top to write this file, it will recreate the agent.py. So this will create the same agent.py file. If you want to see exactly what this looks like in the LangChain implementation, you can check out our documentation on Databricks for the exact workflow and even how to build out multi-agent systems. And just so you know, this full notebook is the same notebook that we'll be using for our lab three. So the rest of the cells in here on how to log an agent are all available. So this is the full orchestration code. The only thing different is if you want to add in your own custom evals. Now let's go into our lab three to finish building this agent, running evaluations, and ultimately deploying it.
