Sign in to continue learning
Context Engineering
7h55m
In the last lesson, we looked at tool calling in the harness. We saw how the harness can search for information and retrieve it for the model. Now, let's jump into the lab, where you will build your own agent. Agents are useful in many application domains, like legal services, where agents can review and analyze documents to help with tasks, such as compliance checks and contract negotiations, or healthcare services, where agents can promote better patient outcomes through better informed decisions or customer service, which will be our focus throughout Module 1 of the course. Context engineering is important for building reliable customer service agents, where establishing trust requires consistently positive user experience. Using agents, customer service representatives can find and access information faster and with less manual review. Asking agents to check refund policies or describe product features or troubleshoot payments. By referring to past transactions without the right information in the right format at the right time, agents might mention a competing company or neglect standard operating procedures. Or forget past interactions with users. Now, in the lab, your agent has access to a data set of customer messages. Since these messages would be private, the model would not have seen them in training. Also, the company would receive new messages all the time, like messages about resetting passwords. As you can see, each entry of the data set is a message about a feature request or a platform issue or a general inquiry. The format is JSON lines, meaning the file is a comma-separated list of messages, with one message on each line of the file, where each message consists of four key value pairs, namely message identifier, the text, user identifier, and a timestamp. You'll find a few messages, like this one about a customer pain point, worth investigating in a real product. The record belongs to a user, Nicholas61. Let's call him Nick, for short. We'll follow Nick across Module 1. Right now, he has three threads open, resetting his password, Slack integration, and syncing after a version update. Suppose we want to find all users bumping into the same problem. After the update to version 5.2 of the product, your agent could search the data set with a regular expression, matching every message that mentions 5.2. Let's see how your agent would handle the task. Remember, the data set of customer messages isn't on a website or a database. It's in the file system. So the agent needs a way to work with files across folders, the same way we do. But an agent doesn't point and click a mouse. It works through the harness, which runs code. So we need a programming language built for the file system. Ideally, one the model already knows well. That way, when the agent needs to read or edit or write a file, it can run different code from the same tool in the harness. The language for this is called Bash. Bash is a programming language, like Python, but for working with a file system. Here we assume a Mac or Linux file system, not Windows. Bash runs in an interface called the command line interface, namely the CLI, also nicknamed the terminal or shell. It's a place to type a command and run it. A few common commands we'll need for the lab include ls, wc, cat, grep, jq, and sed. Let's take a quick look at them. Suppose we have our JSON file with customer messages, along with a TXT file containing the three messages from our customer Nick. Ls lists these two files in our current working directory. Wc counts lines, or words, or characters. We know Nick has three messages. Cat prints the file's contents to the screen. Grep finds every line that matches a regular expression. With a pattern of characters we're looking for, here, only the first message from Nick matches the pattern. Password, jq can search through data in the JSON format, pulling out fields like timestamp from each message. Lastly, sed transforms text by replacing collections of characters, almost like find and replace, on each line of the data set. We can implement a tool to run bash commands, like cat, grep, ls, sed, etc. Since it runs the commands through the shell, we'll call it shell tool. Alongside the tool, we have the system prompt. You are helping the user with the customer support messages file. And user prompt. Who is the customer who experienced issues after upgrading to v5.2? We know the response should include Nicholas61, namely Nick. That's the response you'll get in the lab. Before jumping into the notebook, let's see how the agent will handle the prompt, step by step. First, the model needs to understand the data set. Remember, models have no memory of previous sessions. But the model can tell the harness to run the shell tool with the sed command. Using the flag n with 1200p allows the model to sample the first 200 lines of the data set. The agent doesn't attempt to read everything, just enough to understand the patterns in the data. Particularly, the arrangement of messages, line by line, with fields, docid, text, from email, and timestamp. Since each line of the data set contains one and only one message, including the from email field. The model proceeds to the second iteration in the tool calling loop, where it uses grep, with the flag n to return line number, and i for a case insensitive match. Note the backslash before the period to escape the special character. Remember, the artian grep means regular expression. By running the grep command, the harness retrieves each message from the data set, line by line, containing v5.2, in particular, line 352, which contains the message from that key. Now the model can proceed to the third and final iteration of the loop, where it runs sed command a second time, not for lines 1 to 200, but lines 350 to 355, which contains the message about version 5.2, including the customer email. Finally, the model can provide a response, it's the email address of Nick, namely, Nicholas61. Notice the way the agent handled the task. First, it sampled from the file to understand the organization of the data. Second, it used regular expressions to find line numbers of text related to the user prompt. And third, it extracted the relevant messages to provide a response. This is a common approach to search and retrieval, not just by agents, but by you and me. First, a broad and cursory search, then a narrow and thorough search, and then retrieve the bits and pieces of data for the next steps. Equipped with a shell tool, the agent can search and retrieve data from the file system, like our customer messages. Now, shell tools are useful for several reasons. First, shell tools are general purpose. A few general tools are preferable to many specific tools, because we don't want to build a separate tool for everything. A shell general tool might seem challenging, because we'd have to describe everything it can do in the tool description. The model actually knows about Bash from lots of examples in its training. Second, shell tools are not limited to single commands. They can run many commands in a script provided by the model. Batching multiple commands together reduces the number of iterations in the tool calling loop, which saves time and reduces cost. Third, shell tools can connect to resources, accessible through the command line interface. For example, web browsers are accessible through cURL commands, allowing our shell tool to connect to the Internet. For all these reasons, shell tools are useful to search and retrieve information. But in the next lesson, we will learn that too much information can be just as bad as too little information. So join me there to discuss...