How Did We Go From Chatbots To AI Agents?
The chatbot ecosystem was severely impacted by the advent of Language Models and Generative AI. Incumbents had to figure out how to adapt and incorporate these new technologies. While other companies saw it as a newly paved avenue to enter into the chatbot and Conversational AI world.
Gartner
The disruption was so severe, that Gartner deprecated, or discontinued the Gartner Magic Quadrant for Enterprise Conversational AI Platforms; of which Kore.ai was the leader.
If I had to hazard a guess, I would say that the Gartner Magic Quadrant for Enterprise Conversational AI Platforms will be replaced by a Magic Quadrant for AI Agents.
There will have to be focus on the AI Agent ecosystem, where the AI Agent is hosted and lives. With access to deployment and management tools and easy access to models. Open-sourced and commercial models should be easy accessible, and deployable for an own private instance.
A second element which most probably form part of the criteria is a build framework, where AI Agents can be developed in a no-code to low-code fashion.
Technology Providers
The chatbot technology provider landscape has been very limited and most companies are niche chatbot specific companies which ware founded and exist for the soul purpose of the development and deployment of chatbots or conversational UIs.
There are companies like Oracle, AWS, Google, Microsoft and others which entered the market as a mere extension of their current offering.
However, what made the chatbot / Conversational AI market so compelling is the fact that the smaller niche players were dominating the market to a large degree. With established technology providers like Nuance, Microsoft and others struggling to gain a significant foothold.
Enter AI Agents
With the advent of AI Agents, the ambit of technology providers are sure to increase with companies like ServiceNow and Salesforce entering the AI Agent market. They form part of a group of companies which were not previously considered in terms, or as a part of the chatbot or Conversational AI market.
Together with niche and focussed providers like LlamaIndex, LangChain.
Also consider commercial model providers seeking to expand their reach into applications and getting closer to the end-user. One of the best examples here is the model provider Anthropic who developed a docker framework for a Computer Interface (as seen below).
The Disruption
The disruption to the chatbot ecosystem unfolded gradually at first, with incremental improvements, and then, suddenly, it accelerated rapidly, reshaping the landscape almost overnight.
It Started With Accelerated NLU Development
Large Language Models (LLMs) initially disrupted the chatbot ecosystem by accelerating the Natural Language Understanding (NLU) development process.
Unlike traditional models, LLMs were introduced at design time, not runtime, mitigating risks such as inference latency and aberrant responses in production.
They enhanced NLU by clustering semantically similar customer utterances for intent detection, generating training data, and improving named entity recognition. This resulted in faster and more accurate NLU development.
Then Copywriting & Personas Were Improved
The next phase involved using LLMs for bot copywriting and persona development, again applied at design time.
LLMs assisted developers in crafting consistent and contextually appropriate responses.
Designers could define the bot’s persona, tone, and style, leading to more refined and coherent interactions.
This stage also marked the transition to runtime use, where LLMs started generating dynamic bot responses, handling out-of-domain queries, and augmenting document-based searches.
Advanced Implementations & RAG
Stage three saw the introduction of more sophisticated LLM capabilities at runtime, such as Retrieval Augmented Generation (RAG), document-based Q&A, and prompt chaining.
These implementations allowed for more flexible data formatting, summarisation, and structured outputs.
Prompt chaining enabled longer, complex dialog flows by passing outputs between sequential prompts, enhancing the flexibility and depth of conversational AI.
Overcoming Challenges
When it comes to Language Models, conversational interfaces faced significant challenges, including latency, model drift, catastrophic forgetting, and high inference costs.
Providers like Kore.ai address these issues by enabling easy, no-code deployment of open-sourced models locally, or a hosted private instance.
This helps reduce latency and control inference costs. This local deployment also mitigates model drift and catastrophic forgetting by maintaining consistent, context-specific model versions.
Additionally, they tackle the challenge of creating a contextual reference for models, ensuring more accurate and relevant responses. This holistic approach enhances the performance and reliability of conversational AI systems in real-world applications.
Enter AI Agents
The introduction of AI Agents brought major disruption and opportunity…
Opportunity
With the disruption brought about by AI Agents, created challenges for incumbents, but an huge opportunity for technology providers not previously in this market. Again, consider here the slew of new companies founded on the excitement of AI Agents and also companies like ServiceNow and Salesforce.
Many of the traditional chatbot companies came from a customer care perspective and served as an extension to enhancement to existing customer care or customer experience frameworks.
The new entrants have the advantage, I would argue, of looking at use-cases and implementation scenarios in a new and innovative way.
AI Agents introduced a completely new architecture, which is a quantum leap to the traditional chatbot frameworks.
Disruption
AI Agent architecture has disrupted the landscape so profoundly that chatbot and Conversational AI platform providers must rethink and reengineer their offerings to deliver genuinely agentic implementations.
Many frameworks are being rebranded as AI Agents, though they might lack true agentic functionality under the surface.
The real challenge lies in creating a framework that genuinely supports the development, deployment, management, and extension of AI Agents in a fully agentic manner.
Chief Evangelist @ Kore.ai | I’m passionate about exploring the intersection of AI and language. From Language Models, AI Agents to Agentic Applications, Development Frameworks & Data-Centric Productivity Tools, I share insights and ideas on how these technologies are shaping the future.
