Sign in to continue learning
Nvidia’s NeMo Agent Toolkit: Making Agents Reliable
Intermediate
1h20m
Topics
Agents
It's finally time to deploy your climate_analyzer workflow with a professional web interface. You'll learn how to serve NAT workflows as HTTP and WebSocket APIs. launch the NeMo Agent Toolkit UI and interact with agents in a polished production ready setup. Let's have some fun. Now that we've built our climate agent and we've composed it together with our calculator agent, we are ready to serve it in a production ready workflow. In this lesson, we're going to deploy our workflow as an API. and we're going to connect to the NeMo Agent Toolkit UI to that API and we're going to be able to chat with it in a human interface. We've already seen this command, nat serve. This hosts a FastAPI server and is ready to go with WebSocket support, OpenAPI docs and health checks. An API server is great. Now we can run a UI on top of that and chat with it using natural language. Okay, we built a powerful agent. We've wired in observability, we've run evals against it. Now let's put a UI on top of it so we can actually begin to use it in the real world. In this notebook, we're going to serve our agent as an API. We're going to connect the UI that we saw on the first notebook, and we're going to see what an API first architecture looks like with agents. First, let's install our climate_analyzer just to make sure it's ready to go. As we've seen in previous lessons, normally to serve an API of your agentic workflow is just NAT Serve. But since we're going to be running it in the background in a Jupyter notebook, there's a lot of extra code that goes around it to run it as a subprocess. Don't get too bogged down with that. Just focus on the fact that we're running NAT serve against our config file, like we've seen in the past. And the API server is running. Next, we're going to pull down the NeMo Agent Toolkit UI. The UI is a separate open source repo, and that's because you might have your own UI and we didn't want to cross dependencies. But you can pull it down to your system and run it, which is what we're going to do right now. The NAT repository is pulled and ready to go. So now that we've pulled the UI and we've got the dependencies installed, let's start it up. So on the command line you would run npm run dev, as this is just a standard npm project. Of course, we have to run it as a subprocess in this Jupyter notebook. So we've got it running and it is launched. And here we have some helper script that will give us the proper URL to your UI. It is running locally on port 3000 but depending on where your Jupyter notebook is hosted, the URL might be different. And so this gives us that URL. All right, and let's open up this UI and check it out. And now we're ready to chat with our climate agent. Let's start with one of our suggested questions. What was Mexico's average temperature 1990 to 2000 versus global? You might remember that we looked at this UI in the first notebook and it was just coming up with whatever the LLM happened to have in its training. But now our climate agent has tools that can pull from real world data. And it has access to a calculator so it can do calculations on that data. So let's see how it does with this question. So for our question, we can see the agent successfully ran and gave us the correct answer. And we can also look at the intermediate steps. For instance, we can see the chain of thought as the agent called various tools like calculate_statistics. and the data that it passed into that function and the data that it got from that function. So this is a way for us to evaluate our agents in a live chat environment and we can ask a follow-up question. What was the average temperature specifically in 2000 in Mexico? So we can chat over our data in a more realistic, human setting. Let's ask another question. This question's a little bit more complex. It asks for a complete climate analysis for France with trends and visualization. So we can see it did this complex climate analysis for France and it generated a visualization that it dropped into a repository. The UI is complex and has a lot of additional features. For instance, we can stream results in real time and watch the agent think instead of just getting the results back all in one go. And you can talk to the UI using HTTP streaming or WebSockets or standard HTTP. And this UI is just one example. Any front end can connect to your API. Now that we've taken a look at the UI and ran it against our agent, let's clean everything up so we're ready to go. And we're done. So we have built a climate analyzer. It connects to real-world data sources. It combines multiple agents, one written in LangGraph, another written in plain Python functions. We served it with a production API. We ran evaluations against it, plugged it into observability, and we put a UI in front of that API and proved that we can chat with our agent.
