The Emergence Of Large Language Model (LLM) API Build Frameworks
The real power of LLMs are leveraged by combining & orchestrating LLM functionality with other sources of computation or knowledge.
Before getting into API Build tools for LLMs, let’s first consider traditional chatbot architecture.
We are all familiar with developing chatbots in a web-based GUI where we develop, configure and orchestrate different components to create a chatbot.
These components include NLU, dialog flow management, integration to APIs and more.
Once the chatbot is developed, we expose the chatbot for use via an API.
Below is an example from the IBM Watson Assistant Documentation, showing step-by-step how the Watson Assistant chatbot can be accessed via an API.
The chatbot API can be seen as an intelligent API facilitating user input, and orchestrating different underlying services to yield to the user a response.
Below again is an example from Cognigy where a graphically built application is exposed via a rest endpoint.
So chatbot development frameworks have established this principle of building APIs which are:
- Highly contextual and intelligent
- Orchestrating underlying services and systems
- Creating an unified end-point instead of a dispersed array of different APIs to call
Back To Large Language Models
A product like Retune finds itself more on the side of managing prompts for prompt engineering. But I’m sure it will evolve into more of an orchestration engine as user needs increase.
Current chatbot development frameworks can be divided into the three main categories of:
- Low-Code, &
The LLM Build Tools are also spread across this spectrum.
For instance, LangChain is a pro-code and more technical development environment. Fortunately the documentation and example code are comprehensive and really accelerates the learning process.
Dust has a web based GUI for configuring blocks and chaining these blocks in series or parallel.
Steamship takes as input prompts, pre-existing prompt-chains, and Python code and combines the functionality into a managed API.
Below are some of the LLM, NLP and ASR models Steamship can integrate to.
Retune is more simplistic in their approach, with focus on prompt and chatbot session management together with creating fine-tuned models.
Below you see the Retune model GUI, with the prompt describing the behaviour of the conversational interface. Users can share the model via a REST API.
And below is the LLM chat functionality, with the generated code to create a new chat, creating a new thread, and continuing the conversation.
Functionality will certainly grow within the Retune UI…Retune is not as advanced and granular as Dust or Langchain. However the interface is much less technical and more intuitive for non-technical users.
Humanloop is not generally available yet, but seemingly their product is premised on these principles:
- Prompt sourcing and management
- Fine-tuning custom models
- Managing LLM cost
Existing and established chatbot development frameworks find themselves in a difficult predicament…they need to maintain a firm focus on their product. Continuous augmentation and adding of features to the product is a given due to the investments made to their product.
The new LLM applications have the freedom to play according to LLMs strengths, without inhibitors of an existing framework. You can read more about this predicament here.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
NLU design tooling
“Conversation Designer, Retail, 10k+ employees The tool that turned conversation designers, into NLU designers” ★★★★★…
Get an email whenever Cobus Greyling publishes.
Get an email whenever Cobus Greyling publishes. By signing up, you will create a Medium account if you don’t already…
Eliza Language Technology Community — Language Technology: Conversational AI, NLP/NLP, CCAI…
ELIZA — Where language technology enthusiasts unite.
The Cobus Quadrant™ Of NLU Design
NLU design is vital to planning and continuously improving Conversational AI experiences.
The Cobus Quadrant™ Of Conversation Design Capabilities
∗ This is part one of a two part series, please also take a look part two, the Cobus Quadrant of NLU Design.
Large Language Models Are Forcing Conversational AI Frameworks To Look Outward
With fragmentation being forced on frameworks it will become increasingly hard to be self-contained. I also consider…
Existing Rigid Chatbot Architecture Needs Large Language Model (LLM) Flexibility
In recent posts I have been exploring the impact of LLMs on Conversational AI in general…but in this article I want to…
Large Language Models (LLMs) Will Not Replace Traditional Chatbot NLU…For Now
Traditional NLU pipelines are well optimised and excel at extremely granular fine-tuning of intents and entities at no…
Generative AI & The New Category of LLM Powered Applications
New methods and applications are surfacing to implement conversational experiences by leveraging the power of…
The Anatomy Of Large Language Model (LLM) Powered Conversational Applications
True business value needs to be added to LLM API calls to make any LLM based application successful.