Try out different variants of Linear Quantization, including symmetric vs. asymmetric mode, and different granularities like per tensor, per channel, and per group quantization.
Short CourseIntermediate2 hours 10 mins
Quantization in Depth
Instructors: Marc Sun, Younes Belkada
- Intermediate
- 2 hours 10 mins
- 18 Video Lessons
- 13 Code Examples
- Instructors: Marc Sun, Younes Belkada
What you'll learn
Build a general-purpose quantizer in Pytorch that can quantize the dense layers of any open source model for up to 4x compression on dense layers.
Implement weights packing to pack four 2-bit weights into a single 8-bit integer.
About this course
In Quantization in Depth you will build model quantization methods to shrink model weights to ¼ their original size, and apply methods to maintain the compressed model’s performance. Your ability to quantize your models can make them more accessible, and also faster at inference time.
Implement and customize linear quantization from scratch so that you can study the tradeoff between space and performance, and then build a general-purpose quantizer in PyTorch that can quantize any open source model. You’ll implement techniques to compress model weights from 32 bits to 8 bits and even 2 bits.
Join this course to:
- Build and customize linear quantization functions, choosing between two “modes”: asymmetric and symmetric; and three granularities: per-tensor, per-channel, and per-group quantization.
- Measure the quantization error of each of these options as you balance the performance and space tradeoffs for each option.
- Build your own quantizer in PyTorch, to quantize any open source model’s dense layers from 32 bits to 8 bits.
- Go beyond 8 bits, and pack four 2-bit weights into one 8-bit integer.
Quantization in Depth lets you build and customize your own linear quantizer from scratch, going beyond standard open source libraries such as PyTorch and Quanto, which are covered in the short course Quantization Fundamentals, also by Hugging Face.
This course gives you the foundation to study more advanced quantization methods, some of which are recommended at the end of the course.
Who should join?
Building on the concepts introduced in Quantization Fundamentals with Hugging Face, this course will help deepen your understanding of linear quantization methods. If you’re looking to go further into quantization, this course is the perfect next step.
Course Outline
18 Lessons・13 Code ExamplesIntroduction
Video・4 minsOverview
Video・3 minsQuantize and De-quantize a Tensor
Video with Code Example・11 minsGet the Scale and Zero Point
Video with Code Example・12 minsSymmetric vs Asymmetric Mode
Video with Code Example・7 minsFiner Granularity for more Precision
Video with Code Example・2 minsPer Channel Quantization
Video with Code Example・11 minsPer Group Quantization
Video with Code Example・7 minsQuantizing Weights & Activations for Inference
Video with Code Example・3 minsCustom Build an 8-Bit Quantizer
Video with Code Example・13 minsReplace PyTorch layers with Quantized Layers
Video with Code Example・5 minsQuantize any Open Source PyTorch Model
Video with Code Example・8 minsLoad your Quantized Weights from HuggingFace Hub
Video with Code Example・7 minsWeights Packing
Video・5 minsPacking 2-bit Weights
Video with Code Example・8 minsUnpacking 2-Bit Weights
Video with Code Example・8 minsBeyond Linear Quantization
Video・7 minsConclusion
Video・1 min
Instructors
Course access is free for a limited time during the DeepLearning.AI learning platform beta!
Want to learn more about Generative AI?
Keep learning with updates on curated AI news, courses, and events, as well as Andrew’s thoughts from DeepLearning.AI!