OpenAI Announced 28 Models To Be Switched Off
Here I consider the LLMs which will be switched off, mitigating model deprecation and model drift.
Introduction
A while back OpenAI announced the switching off of 28 Large Language Models.
OpenAI is advising users to migrate all usage to a suitable replacement before the shutdown date. Requests to model APIs past the shutdown date will fail.
This places organisations and application builders at the behest of LLM providers in terms of model availability. The ideal would be for LLM users to become, or at least move towards being model agnostic to some degree, or to self-host models.
The advantage of Gradient Free approaches to Data Delivery is that it is to some degree model agnostic. And it has been proven that In-Context learning at inference acts as a model performance equaliser.
The ideal is for an organisation to be in control of their own destiny by having custom LLM deployment instances.
Operational Disrupters
Apart from model deprecation, cost and LLM Drift are two real service disruptors.
Cost
Apart from models being switched off, there is also the challenge of cost. Cost is calculated in tokens and there are separate token cost for LLM data input and output. Generally the output text cost more than the input text.
And using a model with a larger context window, comes at exorbitant cost, as seen in the graph below.
⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️
LLM Drift
LLM Drift is the definite changes in LLM responses over a short period of time. This is not related to LLMs being in essence non-deterministic or related to slight prompt engineering wording changes; but rather fundamental changes to the LLM.
A recent study found that over a period of four months, the response accuracy of GPT-4 and GPT-3.5 fluctuates considerably in the positive but more alarming…in the negative.
The study found that both GPT-3.5 and GPT-4 varied significantly and that there was performance degradation on some tasks.
Our findings highlight the need to continuously monitor LLMs behaviour over time. — Source
The schematic below shows the fluctuation in model accuracy over a period of four months. it some cases the deprecation is quite stark, being more than 60% loss in accuracy.
More About OpenAI’s Model Depreciation
The models to be depreciated/switched off have been marked at legacy for six moths, prior to the switch off date.
From 4 January 2024, all of these models will be shut-down:
- text-ada-001
- text-babbage-001
- text-curie-001
- text-davinci-001
- text-davinci-002
- text-davinci-003
- davinci-instruct-beta
- curie-instruct-beta
- code-search-ada-code-001
- code-search-ada-text-001
- code-search-babbage-code-001
- code-search-babbage-text-001
- text-search-ada-doc-001
- text-search-ada-query-001
- text-search-babbage-doc-001
- text-search-babbage-query-001
- text-search-curie-doc-001
- text-search-curie-query-001
- text-search-davinci-doc-001
- text-search-davinci-query-001
- text-similarity-ada-001
- text-similarity-babbage-001
- text-similarity-curie-001
- text-similarity-davinci-001
- text-davinci-edit-001
- code-davinci-edit-001
- text-davinci-insert-001
- text-davinci-insert-002
In Conclusion
Apart form the risk of model depreciation, there is also the challenge of managing the non-deterministic nature of LLMs, and LLM Drift…
OpenAI have introduced tools to assist with seeding LLM calls in an attempt to have more control over model responses. Also fingerprint functionality should allow visibility into model changes and updates.
Other challenges are inference timeout, geographic and regional availability.
⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️
I’m currently the Chief Evangelist @ Kore AI. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
