Attention in Transformers: Concepts and Code in PyTorch
- IntroductionVideo・6 mins
- The Main Ideas Behind Transformers and AttentionVideo・4 mins
- The Matrix Math for Calculating Self-AttentionVideo・11 mins
- Coding Self-Attention in PyTorchVideo with Code Example・8 mins
- Self-Attention vs Masked Self-AttentionVideo・14 mins
- The Matrix Math for Calculating Masked Self-AttentionVideo・3 mins
- Coding Masked Self-Attention in PyTorchVideo with Code Example・5 mins
- Encoder-Decoder AttentionVideo・4 mins
- Multi-Head AttentionVideo・2 mins
- Coding Encoder-Decoder Attention and Multi-Head Attention in PyTorchVideo with Code Example・4 mins
- ConclusionVideo・1 min
- Appendix – Tips and HelpCode Example・1 min