In this notebook, you're going to add tracing and observability into the agent that you previously built in the last notebook. So you'll start from the agent that you've already built. And then you're going to add a few things in here to allow you to trace that agent using a rise Phoenix, and then get some better observability into what your agent's doing. So as before, you're going to start out by adding some new imports here that will be used for Phoenix. So importing here Phoenix as well as some basic libraries, and then importing a few things from Phoenix here to set up some of that tracing observability registering here as well as some libraries from Opentelemetry, which you just learned about in the previous slides, and then a library called open inference. Open inference is a library that's used and created by the arise team, and it helps translate some concepts in Opentelemetry to work better with lmms. So those are your libraries there that you're going to be using. Go ahead and import those. And then as before, you're going to set up your OpenAI model and client that you want to use. And now you can launch and connect to an instance of Phoenix. So Phoenix is an application that can receive the traces that you're going to send from your agent here. And then can visualize those in a UI. And so in this environment you've already got an instance of Phoenix running. There are a few ways that you can launch Phoenix. You can either use this command to launch Phoenix inside of a notebook. Alternatively, you can run it on your own local machine, or you can access it through a web app on the arise website. In this case, in these notebooks, Phoenix has been launched for you, so you don't actually have to run anything here. You've already got an instance that's running. Now, Phoenix has a concept of projects, and so projects are used to separate out some of the traces that you're sending. Maybe you're tracing multiple different kinds of applications or agents. You might want to group some of the tracing into different projects. So here you want to define the project name. And then you can use the register method from Phoenix to connect your application here to your Phoenix instance. So this register method here will take in the project name as well as an endpoint. This endpoint is just where your Phoenix instance is running. And that will be where any traces get sent to. So again you have it running in your notebook. And so this helper method here get Phoenix endpoint. It's in the utils function. You can take a look and see what that's doing. If you want to. And that will give you an output that looks something like this where you can see opentelemetry tracing has been set up. And so now any opentelemetry traces that you capture or you instrument will be sent through into your Phoenix project. So next step that you'll go through here is you've made this connection to Phoenix, but you still need to instrument your application and mark which calls and which methods should be sent through into Phoenix. And with what attributes. You can do this manually. However, libraries like Phoenix also have tools to do some of this automatically for you. So this open AI instrument or for example, is part of the open inference library that we mentioned. And what this will do is if I run that, it will take in the tracer provider that we set up here, and it will actually make it so that any calls you make to open eyes library from this point on in the project will get sent through into Phoenix traced properly. So if you're just using open AI calls this would be all you would need to do to actually set up tracing for those calls. In this example agent, you have some stuff beyond that as well. So you have tool calls that get made. You have some other logic that's happening. So you're actually going to combine this automatic instrumented here with some more manual tracing. And so to set up your manual tracing the first thing you want to do here is you want to get a tracer object from that tracer provider. And then this is what's going to be used to mark any of those methods that you want to send through into Phoenix as well. Now with all of that set up, make sure that you've also run the different cells of your agent so that you can then run and test your agent as we trace it. Now when it comes to setting up manual tracing for your agent or really any project, it's always helpful to start from the outermost layer that you want to trace. The outermost trace or span, and then work your way down in terms of the detail that you want to go to. This just helps make sure that you're capturing the right information, and you give yourself a holistic view of what's going on. So in this case, you have your run agent method that's going to be used here to start your agent. However, this method has a loop in it. And as we mentioned in the previous notebook, sometimes you might want to use more of a recursive call here where you're calling this method multiple times for one given run of your agent. So it's helpful sometimes to create an actual another outer layer that you can use, or outer method that you can start as that very top level trace or span here. So in this case you're going to create a new method. In this case the start main span could just be called start agent or whatever you want to use there. And really all this method is doing aside from some of the tracing stuff, is just calling this run agent method that you have already. So it's just calling that run agent method. And again, if you had that as a recursive method, this would be the initial call that you would make to that. I then and then beyond that it's adding in some calls to set up tracing here. So this is your first manual tracing call that's being added. So you'll add in using this tracer object. You'll start the span here with the name Agent Run. You can make the name whenever you want. And then the open inference span kind agent. The span kind is just what kind of category is the span? It'll map to some of the colors and things like that in the Phoenix UI that you'll see in a second. So in this case, agent is the example. You'll see some for tools and for chain later on. And then that will start that current span. And anything within that with clause block will be treated as part of that span. You can also set some attributes for the span. So in this case setting the input to what's the input value for that given span. In this case it'll be the messages object. And then when the agent completes you can set the output here. This will be the value of the return of the agent. And then you can set a status code if you want to to say this was a completed call correctly. There was not an error on this call. So you can define that method. And then from there you can start to work your way down into the agent adding tracer. So next step can be done here is going into your run agent method. And there's maybe a few places that you might want to start spans here. The most obvious one is that each time you call your router, you may want to have a span representing that call. So here's what you can add to do that. So there's been a couple updates to the method here. Everything up here is the same. No changes there as you get down to your while loop. Here's the first change that you'll see. So adding that same call from before tracer start this current span. This time calling it router call and then adding in the open inferred span kind. This time adding in chain as the span kind chain is just a basic logic step. There's no Llvm call or tool call or it's not an agent, it's just a chain change. Almost the default in a way. So you can start that and then add in the input I once again you'll keep your open. I call the same here. Add in your status as well. And then you can add in the output either the tool calls or if you have, all the tool calls completed and you're going back to the user, you can add the, eventual response of your agent here to final response. So you can update that method here, and then you can add a couple more pieces and then take a look at what this looks like in Phoenix. So one other area that you might want to add another span is when you go to handle the tool calls. You might want to have a span that captures that particular method and tracks that method. So here's how you can do that. You can go up to your handle tool call method. And in this case you can use a different technique for actually marking a span. So instead of using the width clause you have another option which is to use a decorator for this method. And this is really useful here because really this method is totally self-contained. If you just said everything in this method can be one span and that would work well for the instrumentation you want to set up. So here you can just mark tracer chain. You could do at trace tracer agent or at tracer tool for the different kinds of tool spans or spans that you would want to track. And then what that will do is it will treat any calls to this method right here as a single span in Phoenix. And then it will take the inputs to this method as the inputs, and then it will take any return from this method as the output value. So if you saw before you were marking the input in the output manually, this is a kind of convenience way that makes it so that you don't have to define those manually. It'll just happen automatically. And so now's a good moment. You've added a few different kinds of instrumentation spans and traces. It's a good opportunity to take a pause, run your agent and see what that looks like in Phoenix. so if you're going to run your agent here, make sure that you're using your newly created start main span method and you can run that. but if you switch over into Phoenix, it should now see you have some data in Phoenix to review. So if you open up Phoenix and to open up Phoenix, there'll be a URL provided in the notebook that you're working in. You open up Phoenix, you'll start on this projects page. And if you remember, you set the project name of Tracing Agent. So you should see there's now one trace inside that you can click into that project. And you should see something that looks like this, where one row here is your trace. You've named it Agent Run. You can click into this and see a few different spans that have been marked and expand this so you can see those. So each row here within your trace is one span. And so your first one is this agent Run. So this was what was set with the with clause inside of start main span. And then you have the router call as a span. And so this was what was set with the width clause inside of your start agent. And then you'll also have this tool called chain span here which is the one that you created using that decorator on your handle tool calls method. You'll also see these orange calls here, which are 11 calls that are made to open AI. And so these are all coming from the automatic instrument that you used at the very beginning of this video, where you added the open AI instrument. So those are automatically getting captured without doing any of the manual instrumentation. And if you expand this a little bit you can see some of the values that are captured here. So starting in your agent run you have your input and output values that are coming there. Similarly for router you'll have your input and output as well. And then if you look at your LM spans you'll have a little bit more information. You'll have things like the user prompt, the system prompt the output. And then you also have all the tools that were supplied. So you can see the tool definitions that were supplied to your agent as well. And in any of these spans you can also always go into the attribute section and see all of the information that was sent to that particular span. So this is where you have all of the info. If you want to really deep dive into some of these. Great. So so far you've got your agent router and your handle two calls method, as well as some of your Lem calls being tracked. But you also have some tools inside of this agent. And it would be helpful to understand when those tools are actually being called and what the responses are there. So you can add in some tracing for those tools as well. So to add tracing to some of your tools, you have a few different options here. First, you can use those decorators that you used for your handle tool calls method. And that's probably the simplest way. So usually the recommended way to go about doing that. So starting here you have your lookup sales data method which is your first tool. And so you can add in a decorator at the top to say tracer dot tool. And then again that will treat this as a single span whenever this method is called. And then it will take the input as this prompt value and the output as whatever the return of this method is. Now this particular tool actually calls some other kinds of methods inside of it. It calls this generate SQL query method. And so it might be helpful to have that as a separate span to say okay, first you want to have a span for generating the SQL code and then a separate one for running the SQL code. So what you can do is if there's part of a method that you want to actually track as an individual span, you can use the width clause that you saw before. So you can add in here before your SQL query is run. You can add in a width clause to say Stardust current span. In this case you can call it execute SQL query and set the open inference span kind as chain once again. And then you'll want to make sure that this is indented so that it's part of that span that you've created there. And then it's would be good to add in some of the input and output values that go along with that as well. So right before this you can add in the input value. And so you can do span dot set input. And in this case the SQL query that's going to be called would make sense as a good input. And then after your call is made you You can say span. Set output and set status. With the result of that SQL query. And in this case you're defining it using the parameter name. You can also just define it by sending the parameter itself whichever way is easier for you to use. And then so now if you run this then that tool will have two spans that it creates whenever it gets run. And we'll have one for the method itself using this decorator here. And then it will have a separate span within that other span nested below for specifically the SQL query running itself. And it makes sense to do that in this case because you might generate correct SQL, but then there might be some connection issue or something else that prevents it from being run correctly. And it's helpful to see where that error might be. So if you see that the internal span here fails but the external one succeeds, then you know that there was an issue with connecting to your SQL database. In this case. And so finally you can add in similar tracing to your remaining tools. And then that'll probably cover all the tracing that you need for the agent here. And so this is fairly straightforward to add in here. Your next method here is pretty simple in this tool. So you can just use the decorator to say trace this as a tool. Again it will grab the input and output for you. Nothing really more that you need to do in this case to trace that particular tool. And then for your data visualization tool again you can use the decorators as well. This is another case where you have a two step tool. But in this case you have multiple steps that are split across two different methods. So you can keep things simple and use the decorators because those contain all of the logic for that particular step. So here for this extract chart config. This in itself is not a tool. This is just part of a tool. So it makes sense to call it chain here. And so you can say at tracer dot chain as opposed to at tracer dot tool. For this extract chart config method. And then you can do the same kind of thing for the other method that gets called here which is your create chart method in your app tracer dot chain. And also make sure that you run any of these cells again as you added things to them just to update that. And then finally you you can go to your last method here this generate visualization and mark this as a tool. So you've marked three different tool spans for your three different tools. You've marked a few different chain spans within those tools. And you've marked your agent and router spans as well. Now you've added all the tracing that you need to so you can go back down and run your agent again, and you should see some more detailed information. And so now if you jump back over into Phoenix, you should see you have a new trace that's been run. So you should now have a second entry. And if you click into that, it'll look slightly different than the one before. You'll have a few more spans here. Notably, you'll have this yellow one of your lookup sales data tool. So that was one that you captured with that decorator on your lookup sales data. And then within that you'll also have a chain span for executing the SQL query. And look at the inputs and outputs of those. this is the one that's been captured using the With clause inside of your lookup sales data method. And if you ran more complicated queries that you used other tools, you would see those tools appearing as well too. So now you have all of the information that you need to understand what's happening inside of your agent at any given step.