The GPT-4 API With 8K Context Is Now Generally Available
This article provides a few examples of how to employ GPT-4 with the simplest Python code, as well as how to utilise the 8K context window for searching and summarising documents larger than 14 pages in length.
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
OpenAI announced that all existing API developers, with a track record of successful payments, can access the GPT-4 API with up to 8K context.
OpenAI describes GPT-4 as their most advanced model. Below a list of the GPT-4 models:
OpenAI is also working on fine-tuning enablement of GPT-4 and GPT-3.5 Turbo which should be available later in 2023.
Below is the gpt-4 model implemented with only a few lines of code:
pip install openai
import os
import openai
openai.api_key = "xxxxxxxxxx"Followed by:
completion = openai.ChatCompletion.create(
model="gpt-4",
messages = [{"role": "system", "content" : "You are a chatbot which can search text and provide a summarised answer."},
{"role": "user", "content" : "How are you?"},
{"role": "assistant", "content" : "I am doing well"},
{"role": "user", "content" : "What is the distance between New York and Montreal?"}
]
)
print(completion)The ChatCompletion endpoint must be used, the following error is given if the default completion endpoint is used (openai.Completion.create):
InvalidRequestError: This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?
And here is the output:
{
"id": "chatcmpl-7akpa51A7KZldPSha2scSslJAzece",
"object": "chat.completion",
"created": 1688994182,
"model": "gpt-4-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The distance between New York and Montreal is approximately 370 miles (or about 595 kilometers) driving distance. The straight-line distance, often called \"as the crow flies\", is about 332 miles or 534 kilometers.\n"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 53,
"completion_tokens": 45,
"total_tokens": 98
}
}Next we submit a document to be summarised using the following code and the user message: You are a chatbot which can search text and provide a summarised answer. Summerise the following text and provide a few key points
completion = openai.ChatCompletion.create(
model="gpt-4",
messages = [{"role": "system", "content" : "You are a chatbot which can search text and provide a summarised answer. Summerise the following text and provide a few key points"},
{"role": "user", "content" : "South Africa, officially the Republic of South Africa, ........The document submitted under the role of user and content is too big, and hence the following error is produced:
InvalidRequestError: This model’s maximum context length is 8192 tokens. However, your messages resulted in 15957 tokens. Please reduce the length of the messages.
After whittling the document down in length until it fits into the context length of 8,192 tokens, I submitted the text block of 4,809 words directly within the Python notebook as context.
The following summarised text is produced, with the output below.
{
"id": "chatcmpl-7alM6JacX7rHFGu0JVZ3BgijNg8tY",
"object": "chat.completion",
"created": 1688996198,
"model": "gpt-4-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "South Africa, officially the Republic of South Africa, is the southernmost country in Africa, bounded on the south by a coastline on the South Atlantic and Indian Oceans. It borders Namibia, Botswana, Zimbabwe, Mozambique, and Eswatini and completely envelops Lesotho. It is a biodiversity hotspot and the world's 24th-most populous nation, home to over 60 million people across diverse ethnic communities. Afrikaans and English are the most dominant European languages spoken in the country.\n\nDespite challenges of widespread poverty and unemployment, South Africa has a significant regional influence and is a member of both the Commonwealth of Nations and G20. It has the second-largest economy in Africa and the world's 39th-largest economy, with the most UNESCO World Heritage Sites in Africa. However, issues persist with crime, corruption, and failing infrastructure.\n\nSouth Africa's history is marked by racial segregation and apartheid, a legacy that has influenced its politics and racial relations. The country became a parliamentary republic and abolished discriminatory laws from 1994, adopting a multicultural national identity known as the \"rainbow nation\". \n\nIn 1961, South Africa became a republic. Although the government strengthened racial segregation, resistance against such policies became increasingly global. This led to international sanctions and divestments. \n\nIn 1990, the National Party lifted the ban on the African National Congress and other political organisations and released Nelson Mandela from prison. This marked the start of negotiations to end apartheid, culminating in the country's first universal elections in 1994, won overwhelmingly by the African National Congress. Today, despite racial tensions and socioeconomic inequalities, South Africa is known for its diverse culture, natural beauty, and rich archaeological heritage."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 6412,
"completion_tokens": 348,
"total_tokens": 6760
}
}And below is the summarised text in document format, 4,809 words are summarised into 268 words:
With the expansion of LLM context windows to 32k and beyond, the autonomy of models will increase. The sheer volume of data which can be submitted in a few-shot approach and interrogated is sure to increase.
What I find interesting is the disposable nature of submitting data directly via a context window and not leveraging a vector database or the like.
⭐️ Follow me on LinkedIn for updates on Conversational AI ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
