Sep 1, 2022
The Large Language Model Landscape
The number of commercial and open LLM providers has exploded in the last 2 years, and there are now many options to choose from for all types of language tasks. And while the main way of interacting with LLMs is still via APIs and rudimentary Playgrounds, I expect that an ecosystem of tooling that helps accelerate their wide adoption will be a growing market in the near future.
Below is a graphic depicting the current Large Language Model (LLM) landscape in terms of functionality, offerings and the tooling ecosystem.
The TL;DR
- Large Language Models (LLMs) functionality can be segmented into five areas: Knowledge Answering, Translation, Text Generation, Response Generation and Classification.
- Classification is arguably the most important to today’s enterprise needs, and text generation the most impressive and versatile.
- The commercial offerings and more general offerings are Cohere, GooseAI, OpenAI and AI21labs. GooseAI currently only focuses on generation.
- The open-source offerings are Sphere, NLLB, Blender Bot, DialoGPT, GODEL and BLOOM.
- The tooling ecosystem is still in a nascent state with many areas of opportunity.
LLM Functionality
Classification
Response Generation
Text Generation
Translation
Knowledge AnsweringThe various LLM offerings cover these five areas of functionality in varying degrees.
Classification is a form of supervised learning where text is assigned to predefined classes. This is related to Clustering which is unsupervised learning where semantically similar text is grouped together without any pre-existing classes.
Response Generation is the notion of creating a dialog flow from example conversations, and having a machine learning approach to it. Where a model determines the next dialog to present to the user, based on the immediate conversation history and the most probable next dialog.
Text Generation can be described as the meta capability of LLMs, text can be generated based on a short description with or without example data. Generation is a function shared amongst virtually all LLMs. Not only can generation be leveraged extensively by few-shot learning data; by casting (prompt engineering) the data in certain way determines how the few-shot learning data will be used.
Translation is where text is translated from one language to another. This is done directly without any intermediary language. Read more about it here.
Knowledge Answering is an implementation of what is called Knowledge Intensive NLP (KI-NLP), where broad domain and general questions can be answered, without querying an API or leveraging a traditional knowledge base. Knowledge Intensive NLP is not a web search, but a self contained knowledge base underpinned by semantic search.
Offerings
Cohere, OpenAI, AI21labs, GooseAI, Blender Bot, DialoGPT, GODEL, BLOOM, NLLB, Sphere
The current commercial offering are constituted by three larger players (Cohere, AI21labs, OpenAI) and an up-and-coming smaller entity in GooseAI.
The open-source implementations tend to be less comprehensive and more specific in their implementation focus.
Tooling Ecosystem
Data-centric Tooling, Playgrounds, Notebooks, Prompt Engineering Tools, HostingLLMs & Playgrounds
LLMs are accessed as APIs, so the barebones tooling required to make use of their APIs is the command-line, a development environment or Jupyter Notebooks; Cohere is doing a really great job of pushing out content that shows how to apply LLMs to real-life use-cases with simple scripts and integrations.
Vendors also clearly realise that to make experimenting and adopting LLMs easier, they need to provide no-code environments in the form of Playgrounds that expose the different tasks and tuning options: these are a great starting point to understand what can be achieved.
Below is the GooseAI playground which is a very similar approach to the other LLM providers.
These playgrounds allow you to play around with "prompt engineering" (which is the way by which you can explore the mind-blowing text generation capabilities). Note: I'm quite surprised that we haven't seen a bigger explosion (yet) of third-party tools / marketplaces etc focused on LLM "prompt engineering", the same way we've seen around image generation models (like DALL-E and more recently Stable Diffusion).
Data-Centric Tooling
I'm anxious to see LLMs more deeply integrated within the "core" workflows required to develop conversational AI and other use-cases like analytics etc; it seems clear that LLM APIs and their embedding spaces are positioned to unlock more powerful:
- Semantic search (useful to explore unstructured data)
- Clustering (needed to identify topics of conversations or intents)
- Entity extraction (via text generation)
- Classification (either via few-shot learning examples, or fine-tuning the actual models)
I don't expect enterprise customers to do this type of work within vendor Playgrounds - instead I expect these will be the types of features incorporated within third-party tools (either the conversational AI platforms themselves, or specialised data-centric solutions) that will be powered by the LLM APIs.
So far, I've only seen HumanFirst integrating LLMs within this type of data-centric offering (and they seem to currently only support Cohere).
🤗HuggingFace
Finally, LLMs are massive models, and they are expensive and difficult to run.
Most of the technologies mentioned here (apart from the commercial LLMs) are accessible via 🤗HuggingFace.
You can interact with models using Spaces, Model Cards or via hosted inference API's. There are options for training, deployment and hosting. Obviously hosting and compute demands will be excessive and not easily justifiable.
In Conclusion
LLMs are not chatbot development frameworks, and the one should not be compared to the other. There are specific LLM use-cases in conversational AI, and chatbot and voicebot implementations can definitely benefit from leveraging LLMs.
