Welcome to Getting Structured LLM Output built in partnership with DotTxt. When working with LLMs in a typical chat interface, unstructured text output is fine, but if you're building software, it becomes much harder to parse and rely on free-form text output. That's where structured output such as JSON becomes very important. They provide a clear format to the machine can read and process. For example, if you want your LLM to look at a product review and generate a couple of fields, say the product name and whether the sentiment is positive or negative, then outputting those two fields in JSON that's downstream software reads that process these fields reliably. In this course, you will learn how to obtain structured outputs using several approaches. The basic idea is you can tell an LLM that you would like the data in a particular format, and if you write the format clearly, specifically using what computer scientists call a regex or regular expression. Then they'll be an efficient way to get an LLM to reliably generate outputs in that format, because regex can be used to enforce constraints on the next token to be generated one token at a time. I'm delighted to introduce the instructors: Will Kurt and Cameron Pfiffer. Will, is a founding engineer at DotTxt, and Cameron is a developer relations engineer. And they help developers like you to build and integrate reliable structured formats into their LLM apps. Thanks, Andrew. In this course you will first get structured output from models that support structured responses. You also learn about the limitations of this approach. To address some of these limitations. We'll use Instructor, an open source library that re-prompts the model until a valid JSON structure is produced. You will also learn how constrained decoding works. this is the underlying concept behind outlines, our open source library that you will use in this course. While learning all the concepts in this course, you'll work on several cool examples and use cases, including a social media analysis agent. The agent reads user posts, identifies the sentiment, and decides whether or not a response is needed. You can even generate a customer support ticket if there's an issue. All of the Json output. After working with retry-based methods, you'll learn how you can get the structure and output from the get-go with outlines, a library that constrains model outputs at the token level by intercepting the logits the probabilities the model assigns to each next token-Outlines blocks any tokens that don't fit your defined schema or format. This guarantees a valid output without any do-overs. In the last lesson, you will learn how you can generate beyond normal JSON. For example, you will learn how to generate valid phone numbers, email addresses, or even ASCII tic-tac-toe boards; and basically any format you can express in a regex. Many people have worked to create this course. I'd like to thank Rémi Louf and Andrea Pessl from DotTxt. From DeepLearning.AI, Esmaeil Gargari has also contributed to this course. The first lesson will be an introduction to structured output generation. That sounds great. Let's go on to the next video and let's get structured.
Getting Structured LLM Output