AI Agent Infrastructure & Build Stack
Here I considering the components required to build, deploy & manage AI Agents.
A few years ago I started mapping out the Conversational AI Technology Landscape…it was a landscape that was expanding in two ways back then…
Firstly, more chatbot development frameworks were introduced on a steady cadence, and the NLU model was the only machine learning component of chatbots.
Much time went into designing conversational flows, and writhing static bot response scripts.
Different approaches were introduced to create some kind of semblance of flexibility when it comes to bot responses and conversational flows.
Innovation focused on improving the NLU model, managing intents with sub-intents and intent hierarchies.
Other developments where detection of digression form the user, and managing it; detecting a change in intent and allowing the conversation to jump into another flow.
Detecting entities (nouns) in a conversation was hard.
Secondly, Large Language Models (LLMs) were introduced which disrupted this set landscape.
And chatbot frameworks started to introduce Generative AI features. It started with detecting and creating intents; generating synthetic NLU training data.
Followed by introducing generative AI nodes in the call flow build UI, where the Generative AI node introduced a measure of flexibility. Prompt chaining became popular and more.
But this was not a complete paradigm shift. But chatbot frameworks had a level of flexibility it did not have before.
Research on Chain-Of-Thought, In-Context Learning (ICL) / RAG and context management were leveraged.
But chatbot development frameworks still found itself in its own little world.
AI Agents a paradigm shift…the reason for this is that AI Agents have the ability to decompose compound user inputs, into a chain of events, on the fly and execute the flow in a step-wise fashion.
Hence the previous two charts became obsolete and really superseded by the new AI Agent landscape. The AI Agent landscape sees new platforms of which LangChain and LlamaIndex are well known.
But here are also a whole host of other software pieces which were introduced. Because the landscape is unfolding at such a rapid pace, the market is fragmented. With start-ups focussing on specific elements which solve for specific challenges, like for instance telemetry.
I’m trying to build this market architecture out, so if you have any suggestions of new categories, or software I need to add, please drop me a note.
Chief Evangelist @ Kore.ai | I’m passionate about exploring the intersection of AI and language. From Language Models, AI Agents to Agentic Applications, Development Frameworks & Data-Centric Productivity Tools, I share insights and ideas on how these technologies are shaping the future.
