Learn to build AI agents with long-term memory with LangGraph, using LangMem for memory management.

Learn how memory works and how the three types of memory - semantic, episodic, and procedural - are used in agentic workflows. Learn how memory works and how the three types of memory – semantic, episodic, and procedural – are used in agentic workflows.Build a personal email agent with routing, writing, scheduling tools to automatically ignore, respond to, or notify about incoming emails. Build a personal email agent with routing, writing, scheduling tools to automatically ignore, respond to, or notify about incoming emails.Add long-term memory to your agent by adding facts, user preferences, and evolving system prompts to a memory store that can be searched for on-going interactions. Add long-term memory to your agent by adding facts, user preferences, and evolving system prompts to a memory store that can be searched for on-going interactions._Learn to build an agent with long-term memory in Long-Term Agentic Memory with LangGraph! Created in partnership with LangChain, and taught by its Co-Founder and CEO, Harrison Chase.Imagine you had a human personal assistant who forgot about their previous conversation with you—not ideal. The same goes for Agents.The main use cases for agents include personal assistance and productivity tasks. One core aspect of these tasks is long-term memory.In this course, you will learn how to build an agent with long-term memory by creating a personal email agent that can respond, ignore, and notify the user using writing, scheduling, and memory tools. You’ll develop your agent’s memory by adding facts to its memory store, providing examples to learn the user’s preferences, and optimizing system prompts to evolve instructions based on previous responses.By the end of this course, you will have the foundational mental framework to build an agent with long-term memory using LangGraph.In detail, you’ll: Learn how the three types of memory–semantic, episodic, and procedural–and the two mechanisms–via hot path and in the background– apply to your agent.  Build an email agent with writing, scheduling, and availability tools, along with a router that triages incoming email and handles it accordingly by ignoring, responding, or notifying the user.Add tools to your email agent that allow it to operate on semantic memory by learning facts about the user, storing them in a long-term memory store, and searching over them in future interactions.Incorporate episodic memory in the form of few-shot examples in the triage step of your agents to help them learn and update user preferences.  Add procedural memory as system prompts, optimized with feedback to improve the instructions the agent follows.Start building agents with long-term memory with LangGraph._It’s helpful to be familiar with Python and a basic understanding of LLM prompting and LLM application development.

Long-Term Agentic Memory With LangGraph

Learn MLOps tools for managing, versioning, debugging, and experimenting in your ML workflow.

Learn to evaluate programs utilizing LLMs as well as generative image models using platform-independent tools Learn to evaluate programs utilizing LLMs as well as generative image models using platform-independent toolsInstrument a training notebook, and add tracking, versioning, and logging Instrument a training notebook, and add tracking, versioning, and loggingImplement monitoring and tracing of LLMs over time in complex interactions Implement monitoring and tracing of LLMs over time in complex interactions_Machine learning and AI projects require managing diverse data sources, vast data volumes, model and parameter development, and conducting numerous test and evaluation experiments. Overseeing and tracking these aspects of a program can quickly become an overwhelming task.This course will introduce you to Machine Learning Operations tools that manage this workload. You will learn to use the Weights & Biases platform which makes it easy to track your experiments, run and version your data, and collaborate with your team.This course will teach you to:Instrument a Jupyter notebookManage hyperparameter configLog run metricsCollect artifacts for dataset and model versioningLog experiment resultsTrace prompts and responses to LLMs over time in complex interactionsWhen you complete this course, you will have a systematic workflow at your disposal to boost your productivity and accelerate your journey toward breakthrough results._Anyone who has familiarity with Python and PyTorch or similar framework and an interest in managing, versioning, and debugging their machine learning workflow.

Evaluating and Debugging Generative AI

Learn how to create an automated CI pipeline to evaluate your LLM applications on every change, for faster and safer development.

Learn how LLM-based testing differs from traditional software testing and implement rules-based testing to assess your LLM application. Learn how LLM-based testing differs from traditional software testing and implement rules-based testing to assess your LLM application.Build model-graded evaluations to test your LLM application using an evaluation LLM. Build model-graded evaluations to test your LLM application using an evaluation LLM.Automate your evals (rules-based and model-graded) using continuous integration tools from CircleCI. Automate your evals (rules-based and model-graded) using continuous integration tools from CircleCI._In this course, you will learn how to create a continuous integration (CI) workflow to evaluate your LLM applications at every change for faster, safer, and more efficient application development.When building applications with generative AI, model behavior is less predictable than traditional software. That’s why systematic testing can make an even bigger difference in saving you development time and cost. Continuous integration, a key part of LLMOps, is the practice of making small changes to software in development and thoroughly testing them to catch issues early when they are easier to fix. With a robust automated testing pipeline, you’ll be able to isolate bugs before they accumulate – when they’re easier and less costly to fix. Automated testing lets your team focus on building new features, so that you can iterate and ship products faster.After completing this course, you will be able to:Write robust LLM evaluations to cover common problems like hallucinations, data drift, and harmful or offensive output.Build a continuous integration (CI) workflow to automatically evaluate every change to your application.Orchestrate your CI workflow to run specific evaluations at different stages of development._Anyone with basic Python knowledge and familiarity with building LLM-based applications.

Automated Testing for LLMOps

Learn how to make safer LLM apps through red teaming. Learn to identify and evaluate vulnerabilities in large language model (LLM) applications.

Learn to identify and evaluate vulnerabilities in large language model (LLM) applications. Learn to identify and evaluate vulnerabilities in large language model (LLM) applications. Apply red teaming techniques from cybersecurity to ensure the safety and reliability of your LLM application. Use an open source library from Giskard to help automate LLM red-teaming methods._Learn how to test and find vulnerabilities in your LLM applications to make them safer. In this course, you’ll attack various chatbot applications using prompt injections to see how the system reacts and understand security failures. LLM failures can lead to legal liability, reputational damage, and costly service disruptions. This course helps you mitigate these risks proactively. Learn industry-proven red teaming techniques to proactively test, attack, and improve the robustness of your LLM applications.In this course:Explore the nuances of LLM performance evaluation, and understand the differences between benchmarking foundation models and testing LLM applications.Get an overview of fundamental LLM application vulnerabilities and how they affect real-world deployments.Gain hands-on experience with both manual and automated LLM red-teaming methods.See a full demonstration of red-teaming assessment, and apply the concepts and techniques covered throughout the course.After completing this course, you will have a fundamental understanding of how to experiment with LLM vulnerability identification and evaluation on your own applications._Red Teaming LLM Applications is a beginner-friendly course. Basic Python knowledge is recommended to get the most out of this course.

Red Teaming LLM Applications

Build systems with MemGPT agents that can autonomously manage their memory.

Build agents with long-term, persistent memory using Letta to manage and edit context efficiently. Build agents with long-term, persistent memory using Letta to manage and edit context efficiently.Learn how an LLM agent can act as an operating system to manage memory, autonomously optimizing context use. Learn how an LLM agent can act as an operating system to manage memory, autonomously optimizing context use.Apply memory management to create adaptive, collaborative AI agents for real-world tasks like research and HR. Apply memory management to create adaptive, collaborative AI agents for real-world tasks like research and HR._Learn how to build agentic memory into your applications in this short course, LLMs as Operating Systems: Agent Memory, created in partnership with Letta, and taught by its founders Charles Packer and Sarah Wooders.An LLM can use any information stored in its input context window but has limited space. Using a longer input context also costs more and causes slower processing. Managing this context window and what to input becomes very important.Based on the innovative approach in the MemGPT research paper “Towards LLMs as Operating Systems,” its authors, two of whom are Charles and Sarah, proposed using an LLM agent to manage this context window, building a management system that provides applications with managed, persistent memory. Examples of Managing Agent Memory are:Control Conversation Memory. As conversations grow beyond defined limits, move information from context to a persistent searchable database. Summarize information to keep relevant facts in context memory. Restore relevant conversation elements as needed by conversation flow.Persist and edit facts such as names, dates, and preferences, and make them available in context.Persist and track ‘task’ specific information. For example, a research agent needs to keep research information in context memory, swapping the most relevant information from a searchable database with previous information.In this course, you’ll learn: How to build an agent with self-editing memory, using tool-calling and multi-step reasoning, from scratch.Letta, an open-source framework that adds memory to your LLM agents, giving them advanced reasoning capabilities and transparent long-term memory.The key ideas behind the MemGPT paper, the two tiers of memory in and outside the context window, and how agent states comprised of memory, tools, and messages are turned into prompts.How to create and interact with a MemGPT agent using the Letta framework, and how to build and edit its core and archival memory.How core memory is designed and implemented with an example of how to customize it with blocks and memory tools.How to implement multi-agent collaboration both by sending messages and by sharing memory blocks.By the end of this course, you will have the tools to build LLM applications that can leverage virtual context, extending memory beyond the finite context window of LLMs._Anyone who has basic Python skills and is curious about how autonomous agents can manage their own memory.

LLMs as Operating Systems: Agent Memory

Move your LLM-powered applications beyond proof-of-concept and into production with the added control of guardrails.

Learn the common failure modes of LLM-powered applications that guardrails can help mitigate, including hallucinations and revealing sensitive information. Learn the common failure modes of LLM-powered applications that guardrails can help mitigate, including hallucinations and revealing sensitive information.Understand how AI guardrails validate and verify your applications with input and output guards, ensuring reliable and controlled interactions. Understand how AI guardrails validate and verify your applications with input and output guards, ensuring reliable and controlled interactions.Add guardrails to a RAG-powered customer service chatbot to create a new layer of control of the applications behavior. Add guardrails to a RAG-powered customer service chatbot to create a new layer of control of the applications behavior._Join our new short course, Safe and Reliable AI via Guardrails, and learn how to build production-ready applications with Shreya Rajpal, co-founder & CEO of GuardrailsAI.The output of LLMs is fundamentally probabilistic, making it impossible to know in advance, or to guarantee the same response twice. This makes it difficult to put LLM-powered applications into production for industries with strict regulations or clients who require high consistency in application behavior. Fortunately, installing guardrails on your system gives you an additional layer of control in creating safe and reliable applications. Guardrails are safety mechanisms and validation tools built into AI applications, acting as a protective framework that prevents your application from revealing incorrect, irrelevant, or sensitive information. This course will show you how to build robust guardrails from scratch that mitigate common failure modes of LLM-powered applications like hallucinations or revealing personally identifiable information (PII). You’ll also learn how to access a variety of pre-built guardrails on the GuardrailsAI hub and are ready to integrate into your projects.You’ll implement these guardrails in the context of a RAG-powered customer service chatbot for a small pizzeria.In detail, you’ll: Explore the common failure modes of LLM-powered applications including hallucinations, going off-topic, revealing sensitive information, and generating responses that can harm your reputation.Learn to mitigate these failure modes with input and output guards that validate and verify your application’s responses.Create a guardrail to prevent the chatbot from discussing sensitive topics, such as a confidential project at the pizza shop.Develop a guardrail to detect hallucinations using a Natural Language Inference (NLI) model, ensuring responses are grounded in the truth of your trusted documents.Add a Personal Identifiable Information (PII) guardrail to detect and redact sensitive information in user prompts and LLM outputs using tools from the Guardrails hub.Set up a guardrail to limit the chatbot’s responses to topics relevant to the pizza shop, keeping interactions focused and appropriate.Configure a guardrail that prevents your chatbot from mentioning any competitors using a name detection pipeline consisting of conditional logic that routes to an exact match or a threshold check with a named entity recognition.The tools in this course will unlock more opportunities for you to build and deploy safe, reliable LLM-powered applications ready for real-world use._Anyone who has basic Python knowledge looking to enhance the safety and reliability of LLM-powered applications with practical, hands-on guardrail techniques.

Safe and reliable AI via guardrails

Learn to build with LLMs by creating a fun interactive game from scratch.

Learn LLM app development by designing and developing a text-based AI game, creating immersive game worlds, characters, and storylines through hierarchical content generation.  Learn LLM app development by designing and developing a text-based AI game, creating immersive game worlds, characters, and storylines through hierarchical content generation. Learn to implement game mechanics using AI to convert text data into structured JSON outputs, enabling features like an inventory detection system. Learn to implement game mechanics using AI to convert text data into structured JSON outputs, enabling features like an inventory detection system.Learn to enforce safety and compliance for AI content creation, and create custom policies using Llama Guard. Learn to enforce safety and compliance for AI content creation, and create custom policies using Llama Guard._Join our new short course, Building an AI-Powered Game, with Together AI and AI Dungeon! Learn from Niki Birkner, Senior Product Manager at Together AI, and Nick Walton, CEO and Co-Founder of Latitude.This course teaches you how to build a LLM-based application by creating your own AI-powered text-based game. You’ll create a world with hierarchical content generation based on a prompt, setting up its core mechanics, and giving it a simple Gradio user interface. By the end of this course, you’ll have a playable text-based game to share with your friends, built from scratch using LLMs. You will be guided through the building blocks of app development using LLMs, including incorporating safeguards using Llama Guard. What you’ll do: Use prompt engineering to create a world with hierarchical content generation so your narrative is consistent and the AI can flesh out your game.Build the first version of your game with a user interface, load the world, and define the core action loop of the game, so you can have a game to play and interact with.Implement safe user-dependent and context-dependent guardrails into your game application with Llama Guard and custom content policy.Learn LLM tool calling by adding story and state components that improve your game’s memory, track inventory, and show progress.  With these techniques, you’ll hone your AI skills by building your own sharable game! _Anyone who has basic Python knowledge and wants to learn how to build with LLMs in a hands-on, fun way!

Building an AI-Powered Game

Learn how to systematically evaluate, improve, and iterate on AI agents using structured assessments.

Learn how to add observability to your agent to gain insights into its steps and know how to debug it. Learn how to add observability to your agent to gain insights into its steps and know how to debug it.Learn how to set up evaluations for the agent components by preparing testing examples, choosing the appropriate evaluator (code-based or LLM-as-a-Judge), and identifying the right metrics. Learn how to set up evaluations for the agent components by preparing testing examples, choosing the appropriate evaluator (code-based or LLM-as-a-Judge), and identifying the right metrics.Learn how to structure your evaluation into experiments to iterate on and improve the output quality and the path taken by your agent. Learn how to structure your evaluation into experiments to iterate on and improve the output quality and the path taken by your agent._Learn how to systematically assess and improve your AI agent’s performance in Evaluating AI Agents, a short course built in partnership with Arize AI and taught by John Gilhuly, Head of Developer Relations, and Aman Khan, Director of Product.When you’re building an AI Agent, an important part of the development process is evaluations or evals. Whether you’re building a shopping assistant, coding agent, or research assistant, having a structured evaluation process helps you refine its performance systematically—rather than relying on trial and error. With a systematic approach, you structure your evaluations to assess the performance of each component of the agent, as well as its end-to-end performance. For each component, you select the appropriate evaluators, testing examples, and metrics. This process helps you identify any areas of improvement so you can iterate on your agent during development and in production. In this course, you’ll build an AI agent, add observability to visualize and debug its steps, and evaluate its performance component-wise.In detail, you’ll: Distinguish between evaluating LLM-based systems and traditional software testing.Explore the basic structure of AI agents – routers, skills, and memory – and implement an AI  agent from scratch.Add observability to the agent by collecting traces of the steps taken by the agent and visualizing the traces.Choose the appropriate evaluator – code-based, LLM-as-a-Judge, and human annotations – for each component of the agent. Set up evaluations for the skills and router decisions of the agent example using code-based and LLM-as-a-judge evaluators, by creating testing examples from collected traces and preparing detailed prompts for the LLM-as-a-judge.Compute a convergence score to evaluate if the example agent can respond to a query in an efficient number of steps.Run structured experiments to improve the performance of the agent by exploring changes to the prompt, LLM model, or the agent’s logic.Understand how to deploy these evaluation techniques to monitor the agent’s performance in production.By the end of this course, you’ll know how to trace AI agents, systematically evaluate them, and improve their performance. _Anyone who has basic Python knowledge and wants to learn to evaluate, troubleshoot, and improve AI agents effectively—both during development and in production. Familiarity with prompting an LLM model would be helpful but not required.