What Is Needed For ChatGPT To Be Enterprise Ready?
What elements need to be introduced for ChatGPT to be used in an enterprise and customer facing implementation? Obviously from a Microsoft / Azure perspective the implementation roadmap can be envisaged. But from a product perspective, what elements will make for ChatGPT to upset the commercial Conversational AI market?
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
For Starters…
There is no doubt that ChatGPT is impressive on so many levels and most implementations/use-cases which have been surfaced on LinkedIn are related to two areas…
1️⃣ The first is using ChatGPT as a personal assistant and having ChatGPT adapt for that audience of one, for a wide range of topics.
2️⃣ The second use-case for ChatGPT which was surfaced was for Microsoft to include ChatGPT in their MS Office suite of products, with specific focus on Microsoft Word.
The other aspect is using ChatGPT in Bing Search, hence upending Google’s monopoly on search. ChatGPT will do well in a Bing-like implementation where vulnerabilities in ChatGPT like having a conversation regarding recent news and events can be supported by Bing.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
Differentiation For Azure
Currently Microsoft is leveraging OpenAI within Azure with OpenAI cognitive services offered by Azure. This point has not been raised much in discussions, but OpenAI is a true differentiator for Azure as a cloud computing solution.
A challenge with commercial Conversational AI implementations is latency and PII legislation which restricts data movement. The question has been raised numerous times in discussion, what about local installations? Well, via Azure with geographic and regional distribution and availability, this is solved for.
Considering Google for a brief moment, Deepmind was acquired by Google in 2014, and who can forget the documentary AlphaGo. Watching that documentary is truly an emotional odyssey as one of the comments on YouTube states. So there is obviously huge AI capability within Google.
But OpenAI has mastered the balance of taking technology to the masses…making everyone feel empowered, instantly turning everyone into AI experts, ruling the time-lines while evading the pitfalls encountered by others.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
ChatGPT As A Product
Creating a finished enterprise ready product necessitates specific requirements…and this is evident in how the OpenAI interface has evolved. Another case in point in terms of a LLM evolving as a product is the Large Language Model (LLM) of Cohere. Significant strides have been made by Cohere in developing a no-code dashboard for LLM management.
🎯 So here is my list of 5️⃣ product requirements for ChatGTP…
1️⃣ Hallucination
It is common for users to say a LLM made something up or lied. This is a common phenomenon with LLMs, the problem of hallucination. This has been raised in papers by Meta AI as a particular problem with LLMs and in the case of Blender Bot 3.
OpenAI GPT-3 is also susceptible to hallucination, hence making up “facts”.
However, the user has the ability the make use of Prompt Engineering and by creating context for the question, the correct answer is yielded by the LLM.
So this is achieved by providing a contextual reference for the LLM to leverage….Below is a simple example of how hallucination can be addressed currently with engineering a generative prompt with context.
Read more about it here.
prompt = """Answer the question as truthfully as possible using the provided text, and if the answer is not contained within the text below, say "I don't know"
Context:
The men's high jump event at the 2020 Summer Olympics took place between 30 July and 1 August 2021 at the Olympic Stadium.
33 athletes from 24 nations competed; the total possible number depended on how many nations would use universality places
to enter athletes in addition to the 32 qualifying through mark or ranking (no universality places were used in 2021).
Italian athlete Gianmarco Tamberi along with Qatari athlete Mutaz Essa Barshim emerged as joint winners of the event following
a tie between both of them as they cleared 2.37m. Both Tamberi and Barshim agreed to share the gold medal in a rare instance
where the athletes of different nations had agreed to share the same medal in the history of Olympics.
Barshim in particular was heard to ask a competition official "Can we have two golds?" in response to being offered a
'jump off'. Maksim Nedasekau of Belarus took bronze. The medals were the first ever in the men's high jump for Italy and
Belarus, the first gold in the men's high jump for Italy and Qatar, and the third consecutive medal in the men's high jump
for Qatar (all by Barshim). Barshim became only the second man to earn three medals in high jump, joining Patrik Sjöberg
of Sweden (1984 to 1992).
Q: Who won the 2020 Summer Olympics men's high jump?
A:"""
openai.Completion.create(
prompt=prompt,
temperature=0,
max_tokens=300,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
model=COMPLETIONS_MODEL
)["choices"][0]["text"].strip(" \n")LLM implementations need to be well engineered and managed from a product perspective.
A complete chatbot can be bootstrapped using LLMs, and this process creating context for the process of generative responses is an important part of the bootstrapped chatbot stack.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
2️⃣ Fine-Tuning
Fine-Tuning is not receiving the attention and traction it deserves, and I see it as the enabler for implementing LLMs like ChatGPT in an organisation.
There needs to be an avenue to create a custom fine-tuned model for a specific implementation in an enterprise. And an enterprise implementation will have a number of custom LLMs for various use-cases.
The challenge is the process of collecting training data, converting the unstructured data into NLU and NLG training data and interfacing to the LLM.
I believe two companies which are addressing and bridging this gap between standard foundational LLMs and creating custom fine-tuned models are HumanFirst and Snorkel AI.
A seen in the image below, the current process for OpenAI GPT-3 of creating and managing embeddings is pro-code and highly technical. This process needs to be addressed by making use of an accelerated, intelligent latent space.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
3️⃣ API Functionality
Obviously ChatGPT will need a programming interface, very much like currently exist for the other GPT-3 models. The playground is where most tinkers spent their time. But the playground can obviously only take you so far, before you will need to step into a more pro-code environment.
Basic GPT-3 parameters being set in Python:
import pandas as pd
import openai
import numpy as np
import pickle
COMPLETIONS_MODEL = "text-davinci-003"
openai.api_key = ("sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")4️⃣ NLU & NLG Design
This point is closely related to fine-tuning…data transformation and manipulation are required for creating embeddings. There is a dire need for a no-code tool to bridge this gap between LLM Data and LLM Models. Data preparation is not only important for NLU Design, but also NLG Design.
For NLG Design there will be a continuous process of updating engineered prompts to assist generative models in generating the most appropriate responses.
I recently wrote a series of articles in an attempt to describe the continuous loop of creating and managing custom LLMs.
5️⃣ Cost
And lastly, cost. OpenAI recently made significant adjustments to their pricing. Currently I’m experimenting with OpenAI with a registered credit card, and thus far the cost is negligible.
A low barrier to entry is important for POC’s and experimentation, this is where Azure and Oracle generally are the least accessible, in my experience.
So there is this balance for OpenAI in charging as much as users are willing to pay, weight up against accessibility. Also the ROI for companies in terms of cost savings/revenue generated matched against ChatGPT pricing.
Here Azure is again a perfect vehicle to productionise, encapsulate and manage GPT-3 related services.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
