Sign in to continue learning
Building and Evaluating Data Agents
Intermediate
Topics
Agents
Let's now add an additional sub agent to the multi-agent workflow. The new agent will search for context from enterprise structured and unstructured data so that we can answer users queries. The data is stored in snowflake and ready for you to use. All right. Let's go. In the last lesson, we built a data agent architecture that looked like this. We had a planner and an executer. Our web researcher gathered data, and then we responded to the user with the chart, generator, chart, summarizer and synthesizer. In this lesson, we're going to expand our data agent's capabilities to retrieve data. We'll do this by adding the cortex researcher agent. The cortex researcher agent is going to access snowflake To retrieve unstructured data using a document search tool and to retrieve structured data using text. NoSQL. This agent will be provided with two tools different than any of the agents we've created already. By adding this agent to our architecture, we can now reason across all of our data. We can ask deep, complex questions that require combining information across internal and external data sources. Now let's get building. Now we're in the notebook. Just like in previous lessons. We'll start by loading the environment. these environment variables still contain access to call LMS and do web search. But we can also connect to snowflake to retrieve data from there as well. In snowflake we've preloaded for you data in databases. So we can understand what structured data looks like. And retrieval for that type of data, along with documents. So we can understand how to retrieve unstructured data. And we've also created a search service that sits on top of that unstructured data. Let's take a closer look at that. Now. To query the data. We'll import our Snowpark session. This is our connection to snowflake. Using the credentials we just talked about. once we've imported the Snowpark session, we can use it and execute SQL directly against it. We'll start by using a specified warehouse that's already been created for you, called the Sales Intelligence Warehouse. You can think about this as the compute that's snowflake uses to execute queries. Then we'll do a second query and select from the sales metrics table. The top five rows. Importantly we'll add this dot collect so that we actually execute the SQL. When we execute the SQL. We actually see the details of all of the information we can get from this table. We see things like did customer names, the deal values, dates. If we've won the deal or not, and even information like the sales rep and the product line associated with the deal. This is the structured data that our data agent will have access to. So we just looked at the structured data. Now, let's take a look at the unstructured data. The unstructured data we have here is meeting notes. Our meeting notes are going to be stored in a table as well. So we can query it again with our snowpark session. They're stored in the sales conversations table in the same schema as the sales metrics. Now we can take a look at an example set of meeting notes. In this example we can see that the meeting with TEC Corp was a discovery call. there was a detailed discussion around integration timeline and complexity. They ask a lot of technical questions and also mentioned a Q2 budget allocation. Overall, the summary says that it was a positive engagement with clear next steps. That's an example of the meeting notes we'll have available to us to help answer questions with our data agent. now that we've taken a look at our data, we can create our cortex agent. We'll start by importing some libraries that we need. We'll import the session from Snowpark along with the route from snowflake. We'll use these to connect our different tools that are already pre-built for us in snowflake. We'll also import some of the same things from long chain and line graph. And then we'll also import this agent. Run the request from snowflake cortex. This is actually how we're going to run our agent that accesses our tools and retrieve unstructured and structured data from snowflake. Now I've mentioned the tools. Let's actually discuss what these look like. So the first tool that cortex agent has access to is cortex analyst. Cortex analyst is a text a SQL where it takes in the user's query and then writes SQL to answer that question. This cortex analyst service relies on a semantic model of what the data actually means. What are the columns themselves mean in the tables, and what are the values mean? Are there any comments and names that are commonly used by people querying this data? All of that information will be included in semantic model file. We've created this for you and loaded into snowflake. We'll just specify exactly the location of the file right here. The second tool that our agent will have access to is the search service. This is a pre-built retriever sitting in snowflake. On top of those meeting notes, that unstructured data that we just looked at. This is a hybrid retriever that uses both semantic search and keyword search, along with Reranking, to return relevant chunks from those documents. Just like the semantic model file, we've already created this and stuff like for you and will simply point to it Here it lives in the sales intelligence database. The data schema and the name of the service is the sales conversation search. So now that we've defined these key resources to use our tools, we can start to initialize and create our cortex agent. Our cortex agent is going to take a single argument, the query that will define in a pedantic base model. Then we'll create our cortex agent tool class. We'll provide a name of the tool. We'll provide a description that it should answer questions using sales conversations and metrics. And we'll give it the schema of the arguments, which is this base model that we just created. We'll initialize our cortex agents tool with the Snowpark session, we'll create a route on top of that session. And then we'll access the cortex agent service that lives inside of that root. Now that we've provided the necessary connection to snowflake for our Cortex Agents tool, we can start to build our request. In our agent for our new request. This is all of the information to make a call to cortex agents. cordless agents is a stateless service that accesses data tools. Tools like the tools we just created for search and text is equal. Well, first provide the model that we want this agent to use. We'll tell it's got three fives on it. And we'll also provide a list of tools analyst and search. Then we'll give each of these tools the tool resources they need to execute and operate. Analyst one. We'll get our semantic model file. This is how the textual sequel actually understands what the data means. Then the search tool we'll get the name of the cortex search service. And we'll provide a number of max results that we want it to return. The last thing is we'll provide a set of messages to the agent. Here that set of messages will just be a single message with our user query. Once we've built the request, we can define a helper to consume the stream. So now that we've built a request, we can create the run method. The run method is going to take in the query and start to execute and return results from our agent. The first step in the run method is to build a request. This is calling the function we just created. And then we'll use the agent service and actually run. Using the request we just built the agent service it's going to return a stream of events. And this stream will contain SQL yet to be executed. If we need to access structured data. And if we're accessing unstructured data, it will return a response along with some citations. To consume the stream, we need to get all of this data out. Let's write a quick helper that's going to allow us to do that. And we'll place this consume stream right above the run method. Consume stream. We'll accept the stream and then respond with text, SQL and citations. Like I said before, for unstructured this will return only the text which is the response. And then the citations. And for structured data we'll get the text which will be a rewritten query along with the SQL yet to be executed. Now that we've defined that helper. We can actually go about executing our SQL. Once we've run the agent service will then consume the stream using the method we just created. Then if we do get sequel, in the case that we need to access structured data, we want to execute, that's equal. To do this well, first use the warehouse sales intelligence warehouse, just like we did when we were exploring the data. And then we'll use the session and execute SQL that was generated from the agent service. To initialize this tool, we'll just instantiated with our Snowpark session. we're going to create a react agent around this tool, just like we did in the prior lesson. our react agent here. It's going to again use GPD four zero and we'll bind it to our tool cortex Agent tool dot run. Then we'll give it a final prompt. We'll tell it it's the researcher. And it's going to perform research using our cortex researcher tool. Once it's found the necessary information, it's going to end its output and not attempt to take further actions. Let's try this agent out here. We're going to try a question that access this is structured data. So we'll ask what are our top three client deals in the background. This agent is going to rewrite that query into something that's really answerable and translatable to SQL. Then it's executing. That's equal to return us the results. Now we can see the top three client deals based on our structured data. Let's try and unstructured example. So here we'll invoke the cortex agent again but will ask what is the next step in the sales process for health tech. This requires information from those unstructured meeting notes. Here we get a detailed answer telling us the three next steps in the sales process to conduct an extensive technical deep dive, provide documentation, and schedule a follow up. This is awesome! Now that we've created our SAP agent, we just need to wrap a graph node around it. That way it's ready to add to our graph. To do this well, extract the agent query from the state, and then we'll invoke our cortex agent using that agent query. We'll add it to our message history. And then we'll go to the executor. Once we're done using the command This is similar to like we did in the web search where we go back to the executor. Now that we've created our additional cortex Agents research node, we're ready to add it to the graph. We defined all the other nodes in the helper method so we can reuse them. And they're defined just as we did before. Then we'll instantiate our state graph just like we did before, and add all of the same nodes with this additional node included the cortex researcher. That's the only change we're making to the cell. Again, to confirm that we built the architecture we intended, we can draw the mermaid graph. And we can see that the cortex researcher is now included in our graph architecture. Now that we've created the graph, let's try it out. So the first question we'll ask we expect to use structured data. We'll ask what are the top three client deals. We'll also ask it to chart the deal value for each. We expect this to use the structured data retrieval tool in Cortex Agents. And then we also expect it to use the chart generation and chart summary nodes as well. So it looks like we've received a response from our agent. Our agent generated a chart displaying nicely the top three client deals with FastTrack and the lead, followed by secure Bank and then Health Tech Solutions. It also provided us a text response where it described exactly what's happening in the data and in the chart. Now let's try some harder queries. This second question that's a little bit tougher. We first asked to identify our pending deals. We research if they may be experiencing any regulatory changes. And then using the meeting notes for each customer we'll ask to provide a value proposition. Given those regulatory changes. This is a lot like the example that Anupam showed right in the beginning of the course. In this example, we asked a tough question that required access to structured data about the deals, web data about any regulatory changes happening in the industries that these companies operate in, along with meeting mode access. To understand our value proposition in each of these deals. Here we can see the response from the agent to this question. The agent was unsuccessful in actually reasoning about this complicated request. It provided some information about how it might go about accomplishing the task. Couldn't actually provide the key details that we need. In the next lesson, we'll be able to learn more about what actually went wrong here. here's the third example that I already ran. Here we asked for a common theme across our different meeting notes. This required access to the unstructured data, And the synthesizer was able to provide us a theme that the meeting notes focused on aligning sales efforts with client needs. In the next lesson, you'll use tracing to really understand exactly what happens under the hood when the agent is executing. You'll also start to use your first evaluations, where you can measure the goal completion of the agent using answer relevance, context, relevance, and groundedness. I'll see you in the next lesson.
Building and Evaluating Data Agents