Welcome to this short course. Built in partnership with Nexusflow and taught by Jiantao Jiao, who is the co-founder and CEO of Nexusflow, as well as assistant professor of ESS stats at UC Berkeley. And Venkat Srinivasan, who is a founding engineer at Nexusflow. Large language models are trained on text, which is a form of unstructured data, but our computing infrastructure is largely built on structured data with virtually defined interfaces like APIs that expect data in a certain format. Function-calling bridges this gap. Here's how LLMs and function-calling works. In the prompt sent to a function-calling capable LLM, will include descriptions of functions that are available to the LLM to use. These descriptions include texts describing what a function might do so the LLM knows when they should use that function, as well as additionally information needed to call that function, such as that function's name and a description of its arguments. When the LLM has a query that it determines is best served by calling a function, the LLM will then generate the needed parameters from the query and return a string that could be used to call that function. Notice that the LLM doesn't call the function directly, it just returns a string that could be used to call that function. These functions are often referred to as tools, and can be used to extend the capabilities of a chatbot, or to build agents, for example, such as a research agent whose tools might include web search or Wikipedia lookups. But function-calling has proven useful in many applications that goes even beyond chat. For example, DeepLearning.AI uses simple AI agents that we build internally to analyze learner feedback to keep on improving our courses. Of course, our core team does read your feedback, and we appreciate you taking the time to provide it. And to gather statistics, we provide an LLM with a prompt that includes a description of a function called "record learner feedback". They can record the sentiment and rating as well as report any technical problems described. The LLM that we used for this is actually a special purpose LLM called NexusRavenV2-13B fine-tuned for function- calling by Nexusflow. Andrew, I'm glad to hear that. NexusRavenV2-13B is an open-source model you can download from HuggingFace. You can also use a hosted version available on our sites. It is a model you'll be using in this course. Many applications do not require a full capability of a general-purpose foundation model. Next, NexusRavenV2-13B has only a certain billion parameters, but can output Chat GPT-4 in some function calling benchmarks. This NexusRavenV2-13B and other fine-tuning smaller models, are small enough to be locally hosted. This eliminates the latency and cost barriers that might prevent you from adding a natural language interface to your applications. There are many instances where you may want to convert natural language inputs, into structured outputs, like a function call. Your business may have a library of functions that perform dashboard operations, and you would like to add a natural language interface. Or you might have applications that need to process transcripts, notes, or proceedings and store them in a database. Every business has its own unique applications. In this course, Venkat will be showing you how you can add a function-calling capability to your applications. We'll start by taking a deeper dive into what function-calling is and how you can use it. You will form prompts with function definitions, as Andrew described, and then you'll use the LLM response to call those functions. Once you have mastered that, we'll kick things up a notch, by defining and calling multiple functions, you will call nested functions, where arguments to one function are themselves functions. Many services on the web have APIs defined using an open API description. You'll also learn to convert these specifications into functions callable by your LLM. You will finish the course with a practical application which takes a customer's service, transcripts and builds SQL calls to store selected data to a database. Many people have worked to create this course. I'd like to thank from Nexusflow, Jian Zhang and Banghua Zhu. From DeepLearning.AI Geoff Ladwig and Esmaeil Gargari, have also contributed to this course. Lots of exciting things with function-calling in this course. Let's go on to the next video to get started, and I hope this course will help you make your applications highly functional.
Function-calling and data extraction with LLMs