Learn how an AI Assistant is built to use and accomplish tasks on computers.

Building toward Computer Use with Anthropic

Learn MLOps tools for managing, versioning, debugging, and experimenting in your ML workflow.

Learn to evaluate programs utilizing LLMs as well as generative image models using platform-independent tools Learn to evaluate programs utilizing LLMs as well as generative image models using platform-independent toolsInstrument a training notebook, and add tracking, versioning, and logging Instrument a training notebook, and add tracking, versioning, and loggingImplement monitoring and tracing of LLMs over time in complex interactions Implement monitoring and tracing of LLMs over time in complex interactions_Machine learning and AI projects require managing diverse data sources, vast data volumes, model and parameter development, and conducting numerous test and evaluation experiments. Overseeing and tracking these aspects of a program can quickly become an overwhelming task.This course will introduce you to Machine Learning Operations tools that manage this workload. You will learn to use the Weights & Biases platform which makes it easy to track your experiments, run and version your data, and collaborate with your team.This course will teach you to:Instrument a Jupyter notebookManage hyperparameter configLog run metricsCollect artifacts for dataset and model versioningLog experiment resultsTrace prompts and responses to LLMs over time in complex interactionsWhen you complete this course, you will have a systematic workflow at your disposal to boost your productivity and accelerate your journey toward breakthrough results._Anyone who has familiarity with Python and PyTorch or similar framework and an interest in managing, versioning, and debugging their machine learning workflow.

Evaluating and Debugging Generative AI

Learn to build six applications powered by vector databases, including semantic search, retrieval augmented generation (RAG), and anomaly detection.

Learn to create six exciting applications of vector databases and implement them using Pinecone. Learn to create six exciting applications of vector databases and implement them using Pinecone.Build a hybrid search app that combines both text and images for improved multimodal search results. Build a hybrid search app that combines both text and images for improved multimodal search results.Learn how to build an app that measures and ranks facial similarity. Learn how to build an app that measures and ranks facial similarity._Vector databases use embeddings to capture the meaning of data, gauge the similarity between different pairs of vectors, and navigate large datasets to identify the most similar vectors. In the context of large language models, the primary use of vector databases is retrieval augmented generation (RAG), where text embeddings are stored and retrieved for specific queries. However, the versatility of vector databases extends beyond RAG and makes it possible to build a wide range of applications quickly with minimal coding.In this course, you’ll explore the implementation of six applications using vector databases:Semantic Search: Create a search tool that goes beyond keyword matching, focusing on the meaning of content for efficient text-based searches on a user Q/A dataset.RAG: Enhance your LLM applications by incorporating content from sources the model wasn’t trained on, like answering questions using the Wikipedia dataset.Recommender System: Develop a system that combines semantic search and RAG to recommend topics, and demonstrate it with a news article dataset.Hybrid Search: Build an application that finds items using both images and descriptive text, using an eCommerce dataset as an example.Facial Similarity: Create an app to compare facial features, using a database of public figures to determine the likeness between them.Anomaly Detection: Learn how to build an anomaly detection app that identifies unusual patterns in network communication logs.After taking this course, you’ll be equipped with new ideas for building applications with any vector database._Anyone with beginner Python, basic machine learning and large language models knowledge who wants to learn applications of vector databases.

Building Applications with Vector Databases

Learn how to easily build AI applications using open-source models and Hugging Face tools. Find and filter open-source models on Hugging Face Hub.

Find and filter open source models on Hugging Face Hub based on task, rankings, and memory requirements. Find and filter open source models on Hugging Face Hub based on task, rankings, and memory requirements.Write just a few lines of code using the transformers library to perform text, audio, image, and multimodal tasks. Write just a few lines of code using the transformers library to perform text, audio, image, and multimodal tasks.Easily share your AI apps with a user-friendly interface or via API and run them on the cloud using Gradio and Hugging Face Spaces. Easily share your AI apps with a user-friendly interface or via API and run them on the cloud using Gradio and Hugging Face Spaces._The availability of models and their weights for anyone to download enables a broader range of developers to innovate and create.In this course, you’ll select open source models from Hugging Face Hub to perform NLP, audio, image and multimodal tasks using the Hugging Face transformers library. Easily package your code into a user-friendly app that you can run on the cloud using Gradio and Hugging Face Spaces.You will:Use the transformers library to turn a small language model into a chatbot capable of multi-turn conversations to answer follow-up questions.Translate between languages, summarize documents, and measure the similarity between two pieces of text, which can be used for search and retrieval.Convert audio to text with Automatic Speech Recognition (ASR), and convert text to audio using Text to Speech (TTS).Perform zero-shot audio classification, to classify audio without fine-tuning the model.Generate an audio narration describing an image by combining object detection and text-to-speech models.  Identify objects or regions in an image by prompting a zero-shot image segmentation model with points to identify the object that you want to select.Implement visual question answering, image search, image captioning and other multimodal tasks.Share your AI app using Gradio and Hugging Face Spaces to run your applications in a user-friendly interface on the cloud or as an API. The course will provide you with the building blocks that you can combine into a pipeline to build your AI-enabled applications!_Anyone who wants to get started building AI applications quickly and easily using open source models.

Open Source Models with Hugging Face

Learn how to quantize any open-source model. Learn to compress models with the Hugging Face Transformers library and the Quanto library.

Learn how to compress models with the Hugging Face Transformers library and the Quanto library. Learn how to compress models with the Hugging Face Transformers library and the Quanto library.Learn about linear quantization, a simple yet effective method for compressing models. Learn about linear quantization, a simple yet effective method for compressing models.Practice quantizing open source multimodal and language models. Practice quantizing open source multimodal and language models._Generative AI models, like large language models, often exceed the capabilities of consumer-grade hardware and are expensive to run. Compressing models through methods such as quantization makes them more efficient, faster, and accessible. This allows them to run on a wide variety of devices, including smartphones, personal computers, and edge devices, and minimizes performance degradation.Join this course to:Quantize any open source model with linear quantization using the Quanto library.Get an overview of how linear quantization is implemented. This form of quantization can be applied to compress any model, including LLMs, vision models, etc.Apply “downcasting,” another form of quantization, with the Transformers library, which enables you to load models in about half their normal size in the BFloat16 data type.By the end of this course, you will have a foundation in quantization techniques and be able to apply them to compress and optimize your own generative AI models, making them more accessible and efficient._This is an introduction to the fundamental concepts of quantization for learners with a basic understanding of machine learning concepts and some experience with PyTorch, who is interested in learning about model quantization in generative AI. 

Quantization Fundamentals with Hugging Face

Build smarter search and RAG applications for multimodal retrieval and generation.

Learn how multimodality works by implementing contrastive learning, and see how it can be used to build modality-independent embeddings for seamless any-to-any retrieval. Learn how multimodality works by implementing contrastive learning, and see how it can be used to build modality-independent embeddings for seamless any-to-any retrieval.Build multimodal RAG systems that retrieve multimodal context and reason over it to generate more relevant answers. Build multimodal RAG systems that retrieve multimodal context and reason over it to generate more relevant answers.Implement industry applications of multimodal search and build multi-vector recommender systems. Implement industry applications of multimodal search and build multi-vector recommender systems._Learn how to build multimodal search and RAG systems. RAG systems enhance an LLM by incorporating proprietary data into the prompt context. Typically, RAG applications use text documents, but, what if the desired context includes multimedia like images, audio, and video? This course covers the technical aspects of implementing RAG with multimodal data to accomplish this.Learn how multimodal models are trained through contrastive learning and implement it on a real dataset.Build any-to-any multimodal search to retrieve relevant context across different data types.Learn how LLMs are trained to understand multimodal data through visual instruction tuning and use them on multiple image reasoning examplesImplement an end-to-end multimodal RAG system that analyzes retrieved multimodal context to generate insightful answersExplore industry applications like visually analyzing invoices and flowcharts to output structured data.Create a multi-vector recommender system that suggests relevant items by comparing their similarities across multiple modalities.As AI systems increasingly need to process and reason over multiple data modalities, learning how to build such systems is an important skill for AI developers.This course equips you with the key skills to embed, retrieve, and generate across different modalities. By gaining a strong foundation in multimodal AI, you’ll be prepared to build smarter search, RAG, and recommender systems._This course is for anyone who wants to start building their own multimodal applications. Basic Python knowledge, as well as familiarity with RAG is recommended to get the most out of this course.

Building Multimodal Search and RAG

Build an interactive system for querying video content using multimodal AI

Create a sophisticated question-answering system that processes, understands, and interacts with complex multimodal data.  Create a sophisticated question-answering system that processes, understands, and interacts with complex multimodal data. Explore the concept of multimodal semantic space and its importance in AI. Explore the concept of multimodal semantic space and its importance in AI.Learn the differences between traditional RAG and multimodal RAG systems, focusing on the complexities of integrating different models. Learn the differences between traditional RAG and multimodal RAG systems, focusing on the complexities of integrating different models._Introduction to Multimodal RAG Systems: Understand the architecture of multimodal RAG systems and interact with a Gradio app demonstrating multimodal video chat capabilities.Multimodal Embedding with BridgeTower: Explore the BridgeTower model to create joint embeddings for image-caption pairs, measure similarities, and visualize high-dimensional embeddings.Video Pre-processing for Multimodal RAG: Learn to extract frames and transcripts from videos, generate transcriptions using the Whisper model, and create captions using Large Vision Language Models (LVLMs).Building a Multimodal Vector Database: Implement multimodal retrieval using LanceDB and LangChain, performing similarity searches on multimodal data.Leveraging Large Vision Language Models (LVLMs): Understand the architecture of LVLMs like LLaVA and implement image captioning, visual question answering, and multi-turn conversations._Anyone with intermediate to advanced knowledge of Python programming, familiarity with machine learning concepts and deep learning frameworks, and a basic understanding of natural language processing and computer vision.

Multimodal RAG: Chat with Videos

Try out the features of the new Llama 3.2 models to build AI applications with multimodality.

Explore the features of the new Llama 3.2 model, from image classification, vision reasoning to tool use. Explore the features of the new Llama 3.2 model, from image classification, vision reasoning to tool use.Learn the details of Llama 3.2 prompting, tokenization, built-in and custom tool calling. Learn the details of Llama 3.2 prompting, tokenization, built-in and custom tool calling.Gain knowledge of the Llama stack, which is a standardized interface for building AI applications. Gain knowledge of the Llama stack, which is a standardized interface for building AI applications._Join our new short course, Introducing Multimodal Llama 3.2, and learn from Amit Sangani, Senior Director of AI Partner Engineering at Meta, to learn all about the latest additions to the Llama models 3.1 and 3.2, from custom tool calling to multimodality and the new Llama stack.Open models are a key building block of AI and a key enabler of AI research. With Meta’s family of open models, anyone can download, customize, fine-tune, or build new applications on top of them, allowing AI innovation. The Llama model family now ranges from 1B model parameters to its 405B foundation model, allowing for diverse use cases and applications.In this course, you’ll learn about the new vision capabilities that Llama 3.2 brings to the Llama family. You’ll learn how to leverage this along with tool-calling, and Llama Stack, which is an open-source orchestration layer for building on top of the Llama family of models.In detail, you’ll: Learn about the new models, how they were trained, their features, and how they fit into the Llama family.Understand how to do multimodal prompting with Llama and work on advanced image reasoning use cases such as understanding errors on a car dashboard, adding up the total of three restaurant receipts, grading written math homework, and many more.Learn different roles—system, user, assistant, ipython—in the Llama 3.1 and 3.2 family and the prompt format that identifies those roles.Understand how Llama uses the tiktoken tokenizer, and how it has expanded to a 128k vocabulary size that improves encoding efficiency and enables support for seven non-English languages.Learn how to prompt Llama to call both built-in and custom tools with examples for web search and solving math equations.Learn about ‘Llama Stack API’, which is a standardized interface for canonical toolchain components like fine-tuning or synthetic data generation to customize Llama models and build agentic applications.Start building exciting applications on Llama!_Anyone who has basic Python knowledge and wants to learn to quickly build on Llama and Llama stack.

Introducing Multimodal Llama 3.2

Learn to use OpenAI Canvas to write, code, and create more effectively in collaboration with AI.

Learn to write and code more effectively with OpenAI Canvas, using features like targeted editing, in-line feedback, changing the reading level, final polish, reviewing code, and fixing bugs. Learn to write and code more effectively with OpenAI Canvas, using features like targeted editing, in-line feedback, changing the reading level, final polish, reviewing code, and fixing bugs.Learn how to build practical use cases like creating a game app, generating Python code from plot screenshots, and creating SQL databases from architecture images. Learn how to build practical use cases like creating a game app, generating Python code from plot screenshots, and creating SQL databases from architecture images.Gain knowledge of how GPT-4o was trained to bring the OpenAI Canvas features and functionalities to life. Gain knowledge of how GPT-4o was trained to bring the OpenAI Canvas features and functionalities to life._Explore a new way to write and code with OpenAI Canvas, a user-friendly interface that allows you to brainstorm, draft, and refine text and code in collaboration with ChatGPT. In the short course Collaborative Writing and Coding with OpenAI Canvas, taught by Karina Nguyen, research lead at OpenAI, you’ll learn how to use this tool to enhance your writing and coding.Canvas provides a side-by-side workspace where you and ChatGPT can collaboratively edit and refine text or code. This makes brainstorming, drafting, and iterating on the text feel more natural and effective.For instance, after writing the first version of your code, Canvas can review it and give suggestions for improvement. It can also help with debugging by adding logs, identifying problems to fix, and writing comments for your code.This course will teach you how to make the most of the new interactive Canvas workspace and tools to make your writing more flexible, more efficient, and a lot more fun as you go behind the scenes with examples of the different use cases and learn what it takes to train the model to powers Canvas’s features and functionalities.In this video-only short course, you’ll:Learn how to ask for in-line feedback and control the iteration of your creative work by directly editing selected areas of your text or code from the model’s output.Learn how to access quick automation tools in a shortcut menu that allows you to modify your writing tone and length, enhance your code, and restore previous versions of your work.Learn how to use Canvas as a research assistant tool with an example of asking the model to reason through the screenshot of a plot to write a research report, in which you can ask questions within the report.Extend its use case by asking the model to write Python code to replicate the graph seen on a screenshot image.Go behind the scenes of how you can create a video game such as Space Battleship from scratch, edit it, and display it in one self-contained HTML file.Get a real-world application example of creating a SQL database from the image of its architecture.Understand the model training and design processes that power Canvas.By the end of this course, you’ll learn about all the possibilities that the new Canvas interface can bring to your writing and coding workflow._Anyone who wants to learn how to use the new OpenAI Canvas interface to write and code better.

Collaborative Writing and Coding with OpenAI Canvas

Learn how to use and prompt OpenAI's o1 model for complex reasoning tasks.

Learn about o1, what makes it work, how it performs, and the best scenarios to use it. Learn about o1, what makes it work, how it performs, and the best scenarios to use it.Learn how to prompt o1 effectively and when to delegate tasks to more cost-efficient, lower latency models. Learn how to prompt o1 effectively and when to delegate tasks to more cost-efficient, lower latency models.Learn how o1 outperforms on coding and vision reasoning tasks, and how to apply meta-prompting to optimize your applications. Learn how o1 outperforms on coding and vision reasoning tasks, and how to apply meta-prompting to optimize your applications._Learn how to effectively prompt and use OpenAI’s o1 model in Reasoning with o1, a short course built in collaboration with OpenAI and taught by Colin Jarvis, Head of AI Solutions at OpenAI.The o1 model is exceptionally good at abstract reasoning tasks. It has record-breaking performance on tasks such as planning, coding, analyzing, domain-specific reasoning like law, and other STEM subjects.In this course, you will learn how OpenAI utilized reinforcement learning to produce a model that uses ‘test-time compute’ to improve performance on many reasoning tasks. Learn what ‘chain-of-thought’ prompting is and how o1 autonomously utilizes it to break problems into smaller steps, try multiple strategies, and think through responses before returning them.In detail, you’ll: Learn to recognize what tasks the o1 model is suited for and when you might want to use a smaller or faster model or combine those two.Understand four key principles of prompting using o1 from “simple and direct” to “show rather than tell,” and explore the difference in performance.Implement a multi-step task in which o1 acts as an orchestrator creating a plan and handing it over to the 4o-mini model to execute the plan in sequence, balancing the trade-off between intelligence and cost.Use o1 for a coding task to build a new application, edit existing code, and test performance by running a coding competition between o1-mini and GPT 4o.Use o1 for image understanding and learn how it performs better with a hierarchy of reasoning, in which it incurs the latency and cost upfront, preprocessing the image and indexing it with rich details so it can be used for Q&A later.Learn a technique called meta-prompting, in which you use o1 to improve your prompts. Using a customer support evaluation set, you iteratively use o1 to modify a prompt to improve performance.Start building applications that require complex reasoning with o1._It’s helpful to be familiar with Python and have a basic understanding of LLM prompting and LLM application development.