DLAI Logo
AI is the new electricity and will transform and improve nearly all areas of human lives.

Welcome back!

We'd like to know you better so we can create more relevant courses. What do you do for work?

DLAI Logo
  • Explore Courses
  • Community
    • Forum
    • Events
    • Ambassadors
    • Ambassador Spotlight
  • My Learnings
  • daily streak fire

    You've achieved today's streak!

    Complete one lesson every day to keep the streak going.

    Su

    Mo

    Tu

    We

    Th

    Fr

    Sa

    free pass got

    You earned a Free Pass!

    Free Passes help protect your daily streak. Complete more lessons to earn up to 3 Free Passes.

    Free PassFree PassFree Pass
Thank you for making it to the end of the course. You now know all the steps to pre-train a model. From preparing your training data and configuring your model, to training it and evaluating its performance. Although the examples you saw here we're running on a CPU, you can edit the code in the notebooks to train larger models on GPUs. Just don't forget to be aware of the costs associated with your pre-training work. With the knowledge from this course in hand, we'd love to see more groundbreaking pre-trained models from you. And please share your work with the community if you do train your own model. We hope you enjoyed the course. Happy pre-training.
course detail
Next Lesson
Pretraining LLMs
  • Introduction
    Video
    ・
    6 mins
  • Why Pre-training
    Video with Code Example
    ・
    12 mins
  • Data Preparation
    Video with Code Example
    ・
    16 mins
  • Packaging Data for Pretraining
    Video with Code Example
    ・
    8 mins
  • Model Initialization
    Video with Code Example
    ・
    16 mins
  • Training in Action
    Video with Code Example
    ・
    11 mins
  • Evaluation
    Video with Code Example
    ・
    7 mins
  • Conclusion
    Video
    ・
    1 min
  • Course Feedback
  • Community
  • 0%