Welcome to this short course on Fast LLM Inference built in partnership with Cerebras. I've long been a fan of Cerebras and so I'm glad that the team is here to teach this course. In this course, you'll build LLM powered applications that respond to user requests quickly by running on hardware designed for fast inference. When an LLM generates text, a big chunk of time is spent moving the model's weights from memory into the compute units. Inference optimized hardware minimizes that movement and makes token generation several times faster than in a typical GPU setup. This is important for agentic workflows which might generate hundreds of thousands of tokens or more before getting back to the user. I'm delighted that the instructors for this course are Cerebras's Zhenwei Gao, Seb Duerr, and Sarah Chieng. Thanks Andrew. We're so excited to be working with you on this one. I remember after you coined the term agentic AI, you also predicted in the batch way back in 2024 that fast inference will be critical to building agents. Many of us in Cerebras were really energized by that prediction. Thank you. During inference, an LLM generates responses one token at a time. For each new token, the model performs matrix multiplications through its transformer layers. And these multiplications depend on the model's learned parameters. On a GPU, the weights don't live physically close to the compute units. They sit in off-chip memory because on-chip memory is too small to hold them all. And so to generate each token, the weights have to travel from off-chip memory to the compute units in chunks. Maybe one layer at a time. And when a model is too large for even one GPU's off-chip memory, then it gets split across multiple GPUs, which then have to exchange intermediate results. This frequent movement of data from memory to compute and between chips drives inference latency. Inference optimized hardware minimizes data movement. Different chip designs approach this challenge in different ways, but one of the best ways is to keep the model's weights on chip and as close to the compute unit as possible. In this course, the hardware you'll be using is Cerebras's wafer-scale engine or WSE3, which takes the approach of making the chip very large. The chip is built from many small cores, units that perform computations and hold on-chip memory. Chips are normally made by cutting a silicon wafer into many small pieces, so that faulty ones can be discarded. The WSE design keeps the wafer intact by routing around defective cores instead. Because the chip is so large, it has enough on-chip memory to hold many models' weights right on the compute units at all times. Take GPT-OSS 120B, a popular open-weight model. In independent benchmarks by Artificial Analysis, the model runs at over 1500 tokens per second on Cerebras, while on typical GPU setups, it runs at roughly a few hundred. Fast inference changes how you build software applications. Instead of hiding latency by adding spinning loaders, streaming responses, or pre-computing results ahead of time, you can build simpler applications. Throughout the course, you'll implement examples of real-time use cases using fast inference. You'll also learn how fast inference changes the way you work with coding agents. When the model generates code in seconds instead of minutes, you can run your tests between tasks instead of saving them for the end. You can keep sessions focused and you can steer the model as it works. So you catch problems early and end up with cleaner code. Many people have worked to create this course. I'd like to thank from Cerebras, Halley Chang, James Wang, Ryan Loney, Chris Ing, Alycia Cary, Joyce Er, and Emma Call. And from DeepLearning.AI Hawraa Salami also contributed to this course. Many of my teams have enjoyed using Cerebras inference and benefit from that for a range of our applications. Please join Zhenwei in the next video where you see why inference speed is becoming more important as you build applications based on agentic reasoning and complex agentic workflows.