A Critical Assessment Of The Gartner® Magic Quadrant™ for Conversational AI Platforms
And Why It Is Important To Understanding The Qualification Criteria
Introduction
The recent Gartner report for Conversational AI platforms caused significant excitement. And for good reason. The companies and platforms recognized as visionary & leading are without a doubt worthy.
To fully appreciate the report within the context of the Conversational AI landscape, the criteria for platforms to qualify for assessment, and have a positive assessment, need to be studies.
These criteria are all listed within the report, a few elements of qualification, or disqualification are:
- The company has to offer an end-to-end, singular stand-alone platform for developing Conversational AI solutions. Hence covering at least the four pillars of chatbots: Intents, Entities, Dialog State Management, Bot Responses/Script.
- Proven voicebot capabilities. This can be tricky, as speech components like Speech-To-Text (ASR) and Text-To-Speech (Synthesized Speech) are highly specialized, requiring large amounts of training data and typically third party vendors are used for this.
- Hence, the offering may use third-party components, as long as they are integrated into, and administered by the platform the vendor is providing. I am always wary of a solution which is dependent on underlying licensed software. The solution is vulnerable to spikes in cost, architectural or functional changes in the underlying components and roadmap updates and changes in licensing terms.
Above, IDC MarketScape: Worldwide Conversational Artificial Intelligence Software Platforms for Customer Service 2021 Vendor Assessment.
General Observations
- The minnows upset the behemoths. For good reason the report saw the likes of Kore AI and Cognigy upset the status-que, and here we are thinking in particular of IBM Watson Assistant, Oracle, Google & AWS.
- The recognition received by these disruptive start-ups are well deserved. With significant focus on an end-to-end solution, improving the dialog state management situation and building structure into intents and entities and more. Certainly worthy in all respects.
- There is an opportunity in democratizing access to Conversational AI, leveraging no-code to low-code. This is evident with Cognigy, where their interface enables a no-code approach. But then have low-code sections and facilitates pro-code to a large extend.
- Looking at a technology like IBM, they have many other complementary services, like Watson Knowledge Studio, Watson Discovery, NLP API, STT, TTS etc. All these services can be used to bolster the Assistant offering.
- AWS & Oracle, and some of the other niche players fall into what I like to call, the “use the cloud you are in” category. Their diminished ability to execute is bolstered by the organization having multiple services and products in a particular cloud. Hence making use of the could their in’s Conversational AI services negates the consideration around execution. More about this later…
- IBM is the leading visionary, but lagging in it’s ability to execute. Watson Assistant is visionary considering the ecosystem they are building, that is undeniable. The lack of execution rating might be a bit harsh, considering the restructuring IBM has gone trough the last coupe of years to address accessibility and customer success.
- Kore AI has strong NLU concepts in their tooling, like sub-intents, and sub-entities. Relationships between sub-entities
- Kore AI has innovative approach to building dialogs with their Conversation Driven Dialog Builder, with a conversational first approach. This reminds somewhat of Rasa’s RASA Interactive Learning and Conversation Visualization.
Caveats To Consider
- There were two noteworthy criteria set for qualification. The one was a definite revenue threshold, which makes sense to some degree. As Naval Ravikant says, the world is an efficient place. And a measure of product efficiency and usability is market adoption, and their willingness to pay for it. This does not mean that there are exceptional platforms which did not qualify; yet.
- One could argue in principle there are seven vertical vectors of conversational AI (more about this later). Niche technologies are focusing on servicing one or two of these verticals as a business. Here an organization like HumanFirst comes to mind. Obviously these organizations were not considered, as they do not offer an end-to-end conversational AI ecosystem, which leads me to the next point…
- The last criteria, and most excluding, is that the conversational AI solution had to be cohesive and complete in their offering. If a framework did not offer a singular stand-alone platform, it did not qualify for inclusion. Most notably, this excluded Microsoft, NVIDIA Riva.
- Not having a singular stand-alone platform is not a weakness or vulnerability…on the contrary. Focusing on a single vertical vector of Conversational AI is not a weakness, but a focused strength. Microsoft & NVIDIA Riva afford organizations the freedom to build their own framework and platform. To create their own underlying structure, do their own integration, and choose the correct solution and software for each component of their augmented conversational AI landscape. This is a strength and deliberate strategy of Microsoft and NVIIDA Riva, and they are playing to it.
- Microsoft has in its arsenal one of the most astute NLU platforms in LUIS. Their STT and TTS (Neural voice) are right up there, if not the best in accessibility. Add to this Azure Bot Service and Bot Framework and Power Virtual Agents. But, because Microsoft does not have a singular stand-alone platform , it did not qualify.
- Ease of use was obviously a big criteria. And in the case of Watson Assistant, Cognigy and the like leads the way in ease of use. Watson Assistant’s new interface in specific is a case in point, of plotting out a roadmap for anyone to build a bot. Much functionality in the new Watson interface is aimed at shielding users from jargon, complexity and configuration intricacies. Even though this is immensely important, this criteria impacted Rasa negatively. Rasa often forms the enabling underlying technology for commercialized platforms.
- Complete end-to-end stand-alone platforms do not inevitably have to seen as a monolith. Cognigy comes to mind here, all their components can used in a stand-alone mode; dialog management, NLU, orchestrating external NLU API’s etc. etc.
Use-the-Cloud-You’re-In
Amazon Lex with Oracle Digital Assistant (ODA) find themselves in this group. My sense is that someone will not easily opt for ODA or Lex if they do not have an existing attachment with Oracle Cloud or AWS from a cloud perspective.
Especially if the Existing attachment is Oracle Cloud or Oracle Mobile Cloud Enterprise. Or with AWS via Echo & Alexa.
ODA & Lex can to some degree operate as a stand-alone solution. It is however most often reliant on other elements of their respective cloud portfolios.
Another impediment with ODA is cost. Free access plays a huge role in developer adoption and the platform gaining that critical mass. We have seen this with IBM being very accessible in terms of their free tier with an abundance of functionality.
Microsoft has gone a long way in more accessible tools, especially with developer environments. RASA has invested much time and effort in developer advocacy. Google Dialogflow is also popular and often a point of departure for companies exploring NLU and NLP.
It needs to be noted that Rasa is the only open-source platform finding itself in the report. Rasa have succeeded to build a very loyal base via exceptional advocacy, user groups and conferences.
Vertical Vectors
Gartner did categorize the vendors in three strategic directions:
- Natural-language-portfolio centric
- Business-automation centric and
- User Experience centric
The Seven Vertical Vectors covering training data, dialog management, NLU, personalization, testing & QA, voice enablement and lastly conversation analysis. These verticals are often niche and some platforms target one or more of these vectors.
Hence Microsoft, which is well-known and has a large product portfolio in the Conversational AI space, was not included. Its conversational AI offerings of Power Virtual Agents, Azure Bot Service and Bot Framework do not offer a singular stand-alone platform, so Microsoft did not qualify for inclusion.
The same could be said about NVIDIA Riva, which follows a similar approach to Microsoft. In having AI offerings in TTS, STT, NLP & NLU. But does not have a singular stand-alone platform.
This is not a disadvantageous, but a deliberate strategy to allow enterprises to build custom-made, highly scalable, Conversational AI ecosystems using independent components.
By the same token, there are niche software companies focusing on narrow vectors.
According to a recent Deloitte report, setup challenges, including training data and maintenance, were among the top reasons for not implementing chatbots in enterprises.
With chatbot development the initial approaches seemed to have been centered around a single vendor or technology framework/stack to address the complete chatbot development and maintenance process, end-to-end.
There is an approach emerging now of building a chatbot ecosystem using various technologies and components. Following a more open approach and opting for the best of breed for each individual component of the conversational experience.
This principle is evident in the open architecture of Cognigy where each component of their framework can be used and implemented independently.
Conclusion
As Conversational AI tools polarize under the seven vertical vectors, or focus on one or more of Gartner’s three strategic directions, doing a apples-with-apples comparison will be increasingly difficult.
