Sign in to continue learning
LLMs as Operating Systems: Agent Memory
Intermediate
1 hour 22 mins
Topics
Agents
LLMOps
Prompt Engineering
RAG
In this lesson, we'll explain the relationship between external memory and RAG. We'll also implement agentic RAG by copying data directly into the agent's archival memory. Let's get coding. We learned in previous lessons that MemGPT agents can use their external memory. So both recall an archival memory for agentic RAG. So this basically means retrieving information from archival and recall to inform the new messages that it generates. Unlike traditional RAG, agentic RAG allows agents to define both when and how to retrieve the data. So, for example, with archival memory search, it also decides what the actual query into the search function should be. So you'll often see variations in what the actual query that the agent makes is. So for example here, it shows the query dog. In MemGPT, agents have a general purpose archival memory that is the default external memory for the agent. So data can be saved in here by both the agent and the user to be used for RAG. So for example, rather than the agent explicitly storing memories, you as a user can do things like upload files like a handbook PDF into the archival memory. Agents can also have additional forms of external memory or retrieve sources for RAG using tools. In this lab, we're going to implement an agent that has access to additional tools for RAG. We're going to go over an example of accessing a fake external database. So once again, we're going to start with importing in our Letta client and initializing the client. And then also using the same helper function that we did before. So in this lab we're going to go over how to implement agentic RAG in Letta or basically agents which can connect to external data sources. In Letta there's two different ways you can do this. One is to copy data into archival memory. We saw before how you could actually insert memories yourself as a developer into the agent's archival memory for the agent to later access. Another way to do it is to connect the agent to external data through a tool. So for example, if you have your own RAG database, you can create a tool that can also access keys. To basically query that data from your agent. So we're going to go over both approaches today. So, in Letta we have a notion of a data source. This is basically a group of embedded data, or set of passages. That we can copy into agents. So I'm going to create a source by creating a source called employee handbook, which basically has the embedding model from OpenAI associated with it. So a source has a single embedding model with it because we can't have inconsistent embeddings, within a single source or within a single agent's archival memory. So I've now created that source. And then I'm now actually going to upload a file into this source. So, to do this is I can upload this file. To a specified source ID. And then we have a local file called Handbook dot pdf, which we're going upload. This upload is basically going to return back a job which will have a certain status associated with it. So right now this job has the status created. But we can actually sort of pull this job to basically see updates over time. So to pull this job, we can have a while loop that checks for the job status to be completed. So here we actually have this while loop that's checking for the status each time and printing it and then sleeping for a second After retrieving the job. So here it actually happened to be that it just completed right now. But you'll see also like a running status like it's completed if your job is taking a bit of longer time. And then once the job is completed, we can actually also print out, the job metadata. So this will have information like the number of passages that were derived from this job. So here we got 11 passages from this document and then under number of documents associated. So in this case it was just one document. And we can also actually list the passages inside of the source. So this will actually have the embedding as well. So I'm not going to print out the whole set of passages. But you can look into the passages text. Or other fields if you're interested. But yeah, you can see here we list out the source passages and basically 11 passages back. So now let's actually just create an agent. In creating this agent, we're going to need to make sure that the embedding model is the same as the embedding model for the source. Otherwise, we're going to have issues if we try to connect the two together. So we create this agent and then now we actually want to attach the source to the agent. So we can do this by basically calling the attach function, and specifying the source ID that we want to attach, and the agent ID that we want to attach it to. And so now we can actually list all the sources that have been attached to an agent. And so you can see that the source that we just created with its ID and then also details about its embedding config, provided back into this list. We can also list the passages from the agent. So you'll see here that the agent now has 11 passages and it's archival memory. Because the passages from the source were copied into the agent's archival memory. So now we can message the agent and ask it to reference data its archival memory from this source. So this source was an employee handbook, and we can tell the agent search archival for the company's vacation policies. So we can see that the agent called archival memory search, with the query vacation policies And this returned back a lot of responses. You can basically see the content that was being returned. And this is return back into the context window. And then the agent pulls out data from this. And returns back with this message which says: "Vacations are permitted only under the following conditions. You must provide an AI agent that matches or surpasses your own competencies, to perform your duties during your absence." So this company is very strict about their vacation policies unfortunately. So another way that we can connect an agent to external data is through tools. So oftentimes, you might have your own database. Maybe a RAG database, or maybe some internal search tool that allows both you or your agents to search some information that you have. So to show an example of this, we can create an agent that has a custom tool that allows it to search some fake external database. So to do this we're going to create, basically a dummy tool that queries into a birthday database. We're going to give it a name. It'll look up the provided names birthday by looking at this fake database. So, in this case, our our database is just a dictionary but you can imagine that this could be a real database where you do a real lookup. And so we will create this tool as birthday tool. And now we can create an agent that has access to this tool. So to do this we're going to create an agent that has human memory block. The human's name is Sarah. And then also persona memory block will give it some extra instructions, telling it you are an agent with access to a birthday db tool. You can look up information about user's birthday using this tool. And then we'll also specify the model name, the embedding name, and then also pass in the birthday tool ID. One other thing that you can do if you are actually connecting to a real database, is you can actually specify per agent tool execution environment variables. So oftentimes with a database or any kind of search API, you have API keys or secrets that you need to pass in. And so if you pass in this variable you can send things into the agent's tool execution environment that can have access to this kind of data. So, obviously since we have a fake database, we don't actually need this. But if you are connecting to a real database, this might be something that you want to do. So now we can actually message the agent and ask it when is my birthday? And it should be able to figure out, using its tool. What my birthday actually is. So we can see that the agent is reasoning, and it calls the query birthday database tool. And request heartbeat equals true. It gets back a birthday. And calls the send message tool to say your birthday is July 6th, 1993. So this was obviously a very simple dummy example, but you can imagine how you could write custom database connectors or custom queries. And connect them to your agent. So congratulations, you've now learned how to connect your agent to custom external data sources, both through the data sources concept inside of Letta and also through defining custom tools. We hope that you can figure out how to add interesting data sources to your agent to make sure your agent has as much context and information as possible.
Week 1: LLMs as Operating Systems: Agent Memory