Loading...
In this lesson, you will learn to use the AI21 SDK, especially using the documents parameter to work with long documents. All right, let's have some fun. Similar to other LLM APIs, Jamba Model API requires two input parameters. First is the chat messages including system message, user message, and assistant message to form a chat history. Second is the model name. You can choose to use Jamba 1.5 Large or Jamba 1.5 mini to balance the quality, latency, and cost based on your use case requirement. There are also a set of optional parameters to help you to build and customize complex genAI applications. Documents is an unique parameter available for you to attach long documents at an input object in your API call directly. You can also enforce the model response in Json format. Use tools parameter for external function and tool calling. Maximum output token counts. Temperature, top_p are also tunable for you. Stop sequence can be very helpful in few shots or many shots. In-context learning. You can also choose to generate multiple responses in one API call, and also stream the response one token at a time, instead of waiting for the entire response to be completed. Throughout the course, you will use the Nvidia 10-K annual report from the last two years as your example data set. Each of the report is around 100,000 tokens, or roughly 200 pages. You can really view the full potential of the Jamba model's long context window to process these SEC filing reports. If you are not familiar with SEC filings, don't worry, you will find out the answer by yourself in just a few moments. Now let's go to the notebook and start to build with Jamba. Let's add these two lines to ignore unnecessary warnings. Now let's import our libraries. Here, you will load the AI21 Python client and also chat messages and documents schema from the Python client. You will also need the API key in this lesson. But don't worry, the API key has already been loaded for you. And here you can create an AI21 client as well. Now you're ready to use the Jamba model. Okay, here you have an example of chat messages as input to the Jamba model. You can give your guidance to the Jamba model in a system message. We can explain the task you want to tell model to do in the user message, and you can attach any previous response from the Jamba model in the assistant message. Here giving word just starting the interaction with the Jamba model. We can remove the assistant message. All right. Now you must first request to a Jamba model. Let's ask the Jamba model to explain the SEC filings in one short sentence to us. Now let's send the messages to the Jamba model. Here you can also customize different optional parameters. Here we choose to use the Jamba 1.5 large model, but you can also choose to use Jamba 1.5 mini as a smaller and faster choice. And you can customize maximum output token the temperature which is default at 0.4 and top_p and also stop sequence. You can also customize the stop sequence. The stop sequence can be a list of strings. The response will end when any other stop sequence string is generated, and the stop sequence will not be included in the generated response. For example, including the separator string from your many shots prompt in the stop sequence can help the model stop at the appropriate position. And you can also specify the number of responses in a single call to the Jamba model. Now you can run this cell to see the response from the Jamba model. A few important things to note in the response. First, the main response is in the Assistant Message Content section. So let's have a look with the Jamba model response. We can now learn the information about SEC filings. So SEC filings are formal documents that publicly traded companies in the United States must submit to the Securities and Exchange Commission, SEC. Providing detailed financial and operational information to ensure transparency and protect investors. There you go. Now you know the precise definition of SEC filing documents. Another important thing to note is the number of tokens in the Usage Info section. So here we used 24 tokens in the prompt, 36 tokens in the completion and total of 62 tokens. You can try it for your token consumption with each call to the general model To show the content of the response message, you can use response dot choices zero message dot content to only get the response string. The Jamba model can also generate response in a specified Json format. Now let's ask a Jamba model to give us some information for the top five most common types of SEC filings in Json format, including the full name and description. Now let's send the request to the Jamba model with additional response format parameter to specify the output should be a Json object and to ensure the output consistency, we can also specify the temperature to be zero for consistent output. Now you can print the model output in a nice Json layout. The 10-K form, which is the annual report, will be used across this course. It's also other popular SEC filing types. Also including the 10-q quarterly report and 8-K, and as well as others. Having the Jamba model providing consistent Json format can be very critical for the robustness of your training application. As we have discussed earlier, one key advantage of the Jamba model is the ability to handle long context. AI21 SDK also provides a convenient way to attach long documents in the API call to Jamba model. For the Jamba model will follow your request or answer your question based on the provided documents. To do so, you can first load in two Nvidia 10-K files. You may also download Nvidia 10-K files directly from the Nasdaq website. The loaded text files can be constructed in a list of document objects using the document schema from AI21 SDK content is a text of your file. And you may also add the metadata to further help the Jamba model to use the document intelligently. As example, you may add company name, the type of documents, and also the year of the 10-K filings in the metadata. With the entire two years of 10-K documents, you can ask the Jamba model to generate an HTML table for the information from both years of 10-K filings. Remember that now the entire two years of 10-K documents are included. The Jamba model is processing about 200,000 tokens, so this step might take a while. Great, now you can visualize it with the IPython display module. Now you have this nice table for you to easily see the financial results from both years 10-K filings. And lastly, in the response usage, we can actually see that the Jamba model just helped you to process over 200,000 tokens from the two years of 10-K files. Great. In the next lesson, you will learn about tool calling with the Jamba model. All right, see you there.