Generative AI & The New Category of LLM Powered Applications
New methods and applications are surfacing to implement conversational experiences by leveraging the power of Generative AI
For a number of years I have been trying to make sense of the Conversational AI landscape, and how development frameworks are shaping tooling to address the demands of a conversational interface.
The progression of Conversational AI Framework functionality has been very linear, to say the least.
During the evolution of chatbot development frameworks, functionality and features have been added managing NLU, developing dialog state machines, knowledge bases grew in importance and more...
However, all of these developments were based on the existing framework (which I detail here). And again, it was largely a linear step progression built on existing Conversational AI framework principles.
Even the implementation of LLM functionality has lead to Conversational AI Frameworks becoming more alike in their attempts to differentiate. With LLM functionality complimenting the existing framework, without truly reinterpreting how the framework should look.
Today I want to embark on a journey of exploring of the available generative AI tools and frameworks which are focussed on LLM based conversational applications; and how the framework of these applications look. While considering the following…
1️⃣ LLMs should no be used in isolation…like many organisations are currently doing. Creative & innovative ways should be sought in which value is added to the LLM API call.
2️⃣ Using LLMs in isolation will not create differentiation or a sustainable business case and is not enough to create a truly powerful application.
And one of the reasons for this, is the general accessibility to LLMs and the general familiarity people of all walks of life now have of LLMs.
How do you build a competitive advantage when everyone is using the same models?
䷓ AI Is The Platform
🎯 There is huge potential for companies to disrupt by building applications from the ground up; making use of Large Language Model API’s. LLMs are ushering in a new paradigm of generative based applications.
🎯 There has been concerns of being at the behest of only a handful of LLM providers. There are continued efforts to open-source LLMs and language technologies.
🎯 LLM-based applications will require a framework which will need to run in some environment. Hence the emergence of frameworks like LangChain.
👨🏽💻 The Prompt Is The Program
🎯 Engineered prompts which invoke a LLM can be seen as a program which is compiled and run in real-time.
🎯 Applications will be required to engineer, manage (store, share, etc) and refine prompts.
🎯 Prompts can also be dynamic as with LangChain applications, where context, conversation memory and more is managed by compiling the prompt during the conversation.
🛠️ Use-Case
🎯 The use-case is of paramount importance, and the differentiator for start-ups.
🎯 Opportunities do exist, and need to be discovered in order to capture value. The greater the value added to LLM API calls, the more successful the start-up will be.
🎯 As seen in the image below, startups can own the UX and Prompt Engineering layers based a LLM.
🎯 The greatest opportunities are the areas of productivity, creative assistance and conversational experiences.
💬 Chat Utilities
🎯 The market is moving away from the notion of a meta-bot, where a single overarching conversational interface serves as a point of convergence.
🎯 There is a proliferation of smaller utility chatbot micro-frameworks. For instance FileChat.IO which is a tool to explore documents using artificial intelligence. Simply upload a PDF and start asking questions to your personalised chatbot.
🎯 Another example is a company called ChatBase which has a very similar product.
🎯 And another example, a company that lets you enter a URL, and summarises what the company’s url is all about.
🤖 Disposable Chatbots
🎯 I like to refer to it micro-chatbots, where chatbots are being treated as disposable utilities.
🎯 Chatbots are created for a specific task or instance, and can be discarded once used.
🎯 Content of a disposable chatbot can be a PDF, any document, a website, a conversation transcript between a customer and service representative.
🎯 Semantic search can be performed on text data in a conversational manner.
🔗 Frameworks Are Composable
🎯 New frameworks are approaching Conversational AI by combining foundation LLMs via a method of composability.
🎯 Composability is achieved with dialogs or conversations being seen as inter-relationship components. Various combinations can be selected and assembled for very specific user requirements.
🎯 LangChain is open-sourced software for the creation of LLM based multi-turn dialogs and managing contextual memory in a conversation.
🎯 Another example is for LangChain to be used to create a conversational QnA chatbot interface to the HuggingFace inference API.
💸 Incumbent Technical Debt
🎯 Current Conversational AI Frameworks carry the burden of retrofitting their current development framework architecture with LLMs.
🎯 Seeking out valuable use-cases is a challenge working with the constraints of current systems.
🎯 New frameworks are being imagined and designed by innovators without any burden of existing technical debt or current framework constraints.
🎯 For instance, a company like DUST uses a completely different lingo than traditional chatbot frameworks. They speak about achieving a particular task. And chaining one or more prompted calls to models and external services (APIs or data sources).
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
