Welcome to Module 1 of our course on Context Engineering. If you've had a chance to work with AI, then you've probably found different applications across a variety of tasks, whether that's summarizing a blog, or translating a caption, or extracting numbers from a report. Models know lots and lots of facts, so you might use them to find information, like what is the capital of Canada? Your question is the user prompt, and alongside it, there's the system prompt, which provides rules to the model, like you are a helpful assistant, always answer concisely. Both the user prompt and system prompt sit in the context window. That is all the data available to the model when generating a response. Here, Ottawa, dividing the context window into message types like user and system, allows the model to make sense of different instructions according to their intent. Searching for information is a common use for popular applications like ChatGPT. Actually, a report from OpenAI suggests that nearly half of prompts in ChatGPT have some component of search, like what was the inflation rate, as opposed to prompts that request something, like factor this mathematical expression, or state something, like I'm a project manager. Let's discuss these three kinds of prompts, expressing, doing, and asking. For expressing, we have prompts like, remember that I'm a project manager. Applications such as ChatGPT will remember these statements, but underlying models like GPT will not. Models don't remember previous prompts or previous responses, meaning the model cannot remember anything about you. All that information doesn't actually get stored in the context window, meaning the model cannot remember context from previous turns of the conversation, let alone previous sessions from another day. So if you ask, what are important skills in my role? Then the model might respond, please provide additional information. To capture a record, the application has to store messages in memory, providing the entire conversation history in the context window alongside the system prompt and user prompt. For doing, we have prompts like, factor this mathematical expression. When models handle more complex tasks like math, they employ reasoning to generate better responses, working step-by-step through many operations, sometimes by trial and error. Without reasoning, the model might incorrectly guess, quantity, x plus one, times quantity, x plus six. But with reasoning, the model might make a plan, like, I need to solve for two unknown quantities, then guess and check various values until verifying a solution. The model is thinking out loud to connect problems and solutions. You won't always see these steps in the response. However, the reasoning traces appear in the context window alongside the prompts. Lastly, for asking, we have prompts like, what was the inflation rate? While the model knows about inflation from a few years ago, it probably doesn't know about inflation from a few months ago. The model has two sources of information, information from its training data and information from its context window. The training data sets have lots and lots of information, including economic data. However, there is a cutoff date, usually several months before the release of the model. The model won't have access to recent information. Additionally, the training data sets contain public data as opposed to private data. Owing to copyright laws, privacy policies, and other restrictions, the model will lack private data, leading to a lack of knowledge about specialized information. This creates a limitation where the model might fail to respond with information or, worse, generate incorrect information. So, any missing information must appear in the context window. Thankfully, we don't need to add the inflation data ourselves. Instead, the model can request extra information from tools, namely functions like a web search function and a web fetch function. They can provide the missing inflation rate in the context window alongside the prompts. For example, if you wrote, remember that I'm a PM as opposed to, remember that I'm a project manager, the model wouldn't know whether you meant project manager or product manager, or maybe even prime minister. Models will continue to get more and more capable. However, they will always need context about you and your projects and your organization. Context will always be important for models. But providing the right information at each step of a task can be hard, not to mention formatting the information this session and storing the information for later sessions. So, in the next lesson, we will discuss ways to programmatically handle context for models. That is, we'll learn to write code to help the model use and reuse prompts, memory, traces, and tools. See you there!