DLAI Logo
AI is the new electricity and will transform and improve nearly all areas of human lives.

Welcome back!

We'd like to know you better so we can create more relevant courses. What do you do for work?

DLAI Logo
  • Explore Courses
  • Community
    • Forum
    • Events
    • Ambassadors
    • Ambassador Spotlight
  • My Learnings
  • daily streak fire

    You've achieved today's streak!

    Complete one lesson every day to keep the streak going.

    Su

    Mo

    Tu

    We

    Th

    Fr

    Sa

    free pass got

    You earned a Free Pass!

    Free Passes help protect your daily streak. Complete more lessons to earn up to 3 Free Passes.

    Free PassFree PassFree Pass
In this course, you learned about three types of attention self-attention, masked self-attention, and encoder-decoder attention. You went in deep and explored the strengths and weaknesses of self-attention versus masked self-attention, and learned when to use each one. You also coded the three types of attention and verified that they worked as expected. I'm looking forward to seeing what you will build on your own. Bam!
course detail
Next Lesson
Attention in Transformers: Concepts and Code in PyTorch
  • Introduction
    Video
    ・
    6 mins
  • The Main Ideas Behind Transformers and Attention
    Video
    ・
    4 mins
  • The Matrix Math for Calculating Self-Attention
    Video
    ・
    11 mins
  • Coding Self-Attention in PyTorch
    Video with Code Example
    ・
    8 mins
  • Self-Attention vs Masked Self-Attention
    Video
    ・
    14 mins
  • The Matrix Math for Calculating Masked Self-Attention
    Video
    ・
    3 mins
  • Coding Masked Self-Attention in PyTorch
    Video with Code Example
    ・
    5 mins
  • Encoder-Decoder Attention
    Video
    ・
    4 mins
  • Multi-Head Attention
    Video
    ・
    2 mins
  • Coding Encoder-Decoder Attention and Multi-Head Attention in PyTorch
    Video with Code Example
    ・
    4 mins
  • Conclusion
    Video
    ・
    1 min
  • Appendix – Tips and Help
    Code Example
    ・
    1 min
  • Course Feedback
  • Community
  • 0%