Sign in to continue learning
Context Engineering
7h55m
In the previous lesson, we studied ways that models struggle with long context. Just like we want to avoid too little context, we want to avoid too much context with too little context. The model doesn't have enough information to complete tasks. But with too much context, the model is less affordable, less fast, and less attentive to details. We studied these issues in the last lesson, through a logic puzzle, with too little information. The model could not determine the correct answer of five hours. It hallucinated about the rate of melting, taking it to be zero snowballs every 15 minutes. With too much information, the model has access to the rate of melting, but might struggle to find it, like looking for a needle in a haystack. So, taking it to be zero snowballs every 15 minutes, arrive at the incorrect answer of three hours. Models can get things wrong, and not always the same way twice. There's a certain amount of randomness to the way models work, but run them enough. And the mistakes settle into a few recurring patterns, called failure modes. Say, when a model is unsure, maybe from too little context, it guesses. And the guess can vary from run to run. But once it makes an incorrect guess, that mistake can negatively impact the session, and with memory, impact later sessions as well. We nickname this context poisoning. It's one of three failure modes that we'll study in this lesson. Note that context poisoning can happen with short context, as opposed to long context. The same with our other failure modes, context confusion and context clash, meaning you might bump into them. Even with a relatively low token count, well within the effective context length, say 100,000 or 200,000 tokens, for a 1 million token context length, context clash occurs when there's inconsistent information. That's too dissimilar. Context confusion occurs when there's distracting information. That's too similar. While the example about snowballs might seem silly, we have to remember that agents get information from the user, from the model, and from the harness, which searches and retrieves information from context sources, like websites, databases, and files. When the agent works independently over minutes, or even hours, there's a chance for inconsistent information, or distracting information, to build up in the context, to describe these three context failure modes. In the setting of our running example throughout Module 1, on customer service, let's imagine a scenario that shows context poisoning, context clash, and context confusion. So, imagine an agent for a concert ticket platform that can search policies, look up orders, and check seat availability, among other tools, remembering each customer along the way. We'll take three cases in detail, each with its own customer, starting with our old customer, Nick, then new customers, Ramesh and Maria. Nick asks the agent to refund his tickets, but he leaves out a detail he bought them resale, from another fan, not from the platform, and the agent never asks a follow-up question. It just assumes a direct purchase on the platform, and fills in the rest. Your tickets are eligible for a full refund. That assumption is now in the context, recorded as if true, when an omission or error gets into the context, and gets repeatedly referenced, it impacts later turns of the conversation. We call this context poisoning. The model, user, or harness could be to blame. The user might make a mistake, even leaving out a detail that the agent then fills in with a guess, sometimes a hallucination. Or the harness could retrieve factually incorrect information, from context sources, like unauthoritative websites. One way to address context poisoning is planning and evaluation, which we study in Module 3. Planning could check the agent before it runs, indicating steps to follow, like first, ask about resale, or evaluation could check the agent after it runs, providing validation on responses, like always double-check resale. In Nick's case, evaluation could help the harness search and retrieve the appropriate policy. Next, consider Ramesh, who asks about eligibility for backstage passes. Previously, the ticketing company had a single service tier, but currently, the ticketing company has both basic tier and premium tier, with premium tier offering perks, say, backstage passes to meet the musicians. Among the context sources is the company knowledge base, with policies that are findable and accessible through the search policies tool. While the old policy is gone from the knowledge base, it might be available to the agent, say through memory, which contains snippets of messages from previous conversations. So the agent has to operate on inconsistent context, containing both the old information and new information. We call this context clash. It's why some organizations have a zero-copy policy, where all data must remain in place, so there is a single source of valid data, despite changes over time. Tagging data with metadata, particularly timestamps, is part of change management. We will study versioning context with metadata and forking. In Module 2, both metadata and forking are ways to address context clash. Metadata allows us to add timestamps, like year, month, day, and forking allows us to split data sets into complementary versions, with some shared data and some separate data, say, customer records for premium tier versus basic tier service. Here, we can fix the issue with version metadata, so the agent can tell the fresh policy from the stale policy, with a metadata tag on each policy. The harness takes the most recent, and Ramesh gets his answer. Hopefully, some backstage passes as well. Lastly, consider Maria, who wants a record of her previous order. She asks the agent to pull up the details. The right tool is lookup order, which fetches a single order, but that harness contains many tools, such as update order, which views and changes order details. Both descriptions mention order details, so Maria's word details matches both tools. The model selects update order, the wrong tool. We call this context confusion. When two pieces of information, particularly two tool descriptions, are nearly duplicate, the model cannot reliably distinguish them, and may select the wrong one. Now, there are two issues. The tool descriptions are too similar, and the context window has too many tool descriptions. Update order, lookup order, search policies, etc. There's just too much information in the context window. Remember from previous lessons that models cannot attend to details with too much context. We should address both issues. First, we should distinguish tools through more detailed descriptions. And second, we should use tool registries. In Module 4, we will study tool registries, which are a way to address context confusion, rather than include every tool description in the context window. The harness can search through a registry of tools, filtering a large set of tools, down to a small subset, relevant to a particular task. For Maria's request, the harness searches the tool registry, and selects two tools from among many, lookup order and update order. Now, with just two tool descriptions, the model can then select lookup order, especially with an improved description that details the tool, including constraints of calling it. With all present, the agent performs the lookup Maria asked for. So, to summarize. Context poisoning is an incorrect fact that gets into the context and misleads the subsequent responses. Context confusion and context clash are information that's too similar, or on the other hand, too dissimilar. Remember that these issues can happen with short context, as opposed to long context, well within the effective context length. But certainly issues are more likely to occur with long context, with lots of information. Some from yourself and user prompts, others from the harness and tool output. Eventually, there will be contradictory information, or duplicate information, or maybe both. Because you could have two or more failure modes at the same time, including others. Besides context, poisoning, confusion, and clash. This isn't an exhaustive list of issues. You will bump into specific, affecting your application. But the experience we're gaining with more common failure modes will help us spot less common failure modes before they cause us trouble. Monitoring is an important part of that. If your agent is going in circles or off on tangents, then you need to be tracking the context available to it. Otherwise, you won't be able to identify your own failure modes for your application. So, in the next lab, we add logging to our agent. See you there.