Updated: A Comparison Of Eight Chatbot Environments

Discover Which Chatbot Framework Might Work Best For Your Organisation

Cobus Greyling

--

Introduction

This research based on a nine point comparison matrix, here I look at the strengths, weaknesses and possible growth areas of each solution…

I have built prototypes with most of the commercial cloud and opensource Conversational UI & AI platforms currently available.

General Trends

There are a few general chatbots trends emerging…

  • Intents and Entities are merging and contextual annotation of entities within the intent or utterance is becoming commonplace & very necessary.
  • Data structures are being introduced to Entities. Entities can have sub and sub-sub entities, and can be decomposed. This trend is visible in Rasa, Alexa Conversations tool and Microsoft LUIS. Rasa calls it Entities Roles & Groups. AWS calls it Slots with Properties. And Microsoft LUIS, entities which can be decomposed.
  • With the advancement of AI, platforms and development environments are becoming more complex. Astute environments are moving complexity to the background and presenting the user with a very simplistic yet powerful interface. When OpenAI went from GPT-2 to GPT-3, they introduced an API due to the sheer size and computing power required. As Rasa is advancing in functionality, their UI (Rasa-X)becomes more intuitive and essential. There is an art to it, which they are mastering.
  • Deprecating of the State Machine is inevitable, Rasa is leading the charge here. IBM is introducing more automation to their Dialog Management system.

Overview Of Development Environment

I have found that environments are generally very similar in their approach to tools available for crafting a conversational interface.

Considering what’s available, chatbot development environments can still be segmented into 4 distinct groups.

These groups being:

  • Leading Commercial Cloud Offerings
  • NLU / NLP Tools (mostly opensource)
  • The Avant-Garde
  • The Use-the-Cloud-You’re-In

Leading Commercial Cloud Offering

The leading commercial cloud environments attract customers and users to them purely for their natural language processing prowess and presence.

Among these I would count IBM Watson Assistant, Microsoft Bot Framework / Composer / LUIS / Virtual Agents, Google Dialog Flow etc.

Watson Assistant Logo

Established companies gravitate to these environments, at significant cost of course. These are seen as a safe bet, to meet their Conversational AI requirements.

They are seen as chatbot tools providers in and of their-self.

Scaling of any enterprise solution will not be an issue and continuous development and augmentation of the tools are a given. Resources abound with technical material, tutorials and more.

NLU /NLP Tools

There are also opensource tools like spaCy, Apache OpenNLP, RASA NLU and others which can be used to to process natural language in your environment.

Some organisations are creating their own chatbot framework making use of these tools.

Industrial-Strength Natural Language Processing

This is the harder route and is more time consuming, but if you have an existing environment, augmenting it with natural language processing capability, making use of these tools is a viable option.

It is truly astonishing the power of most of these opensource tools. And with the documentation available, it can serve as a “no software cost” point of departure for a first foray into natural language processing.

With the announcement from OpenAI that GTP-3 will be available via an API, a new tool is introduced which can be leveraged by chatbot frameworks. The extend is hazy, but all at a cost of course.

The Avant-Garde

Here RASA really finds itself alone at the forefront. They follow a very unique path in terms of wanting to deprecate the state machine and hard-coded dialog flows/trees. Together with their Conversation Driven Design (CDD) in the form of Rasa-X this is a very compelling option.

RASA

Their entities are contextually aware and they follow an approach where entities and intents really merging. Compound entities are part of the offering. Entities can be segmented according to roles and groups.

It is said their market-share is currently at 6% (I could not confirm this, I believe it’s more). However based on their expansion, funding, developer advocacy and events, this is a company to watch. And hopefully the bigger players will emulate them. One of their strong points is developer advocacy and being the technology of choice for seed projects.

RASA has succeeded in creating a loyal developer following.

Use-the-Cloud-You’re-In

I cannot help but feel Amazon Lex with Oracle Digital Assistant (ODA) find themselves in this group. My sense is that someone will not easily opt for ODA or Lex if they do not have an existing attachment with Oracle or AWS from a cloud perspective.

Especially if the existing attachment is Oracle Cloud or Oracle Mobile Cloud Enterprise. Or with AWS via Echo & Alexa.

Oracle Digital Assistant

Another impediment with ODA is cost. Free access plays a huge role in developer adoption and the platform gaining that critical mass. We have seen this with IBM being very accessible in terms of their free tier with an abundance of functionality.

Microsoft has gone a long way in more accessible tools, especially with developer environments. RASA, even though a relatively late starter, has invested much time and effort in developer advocacy. Google Dialogflow is also popular and often a point of departure for companies exploring NLU and NLP.

ODA is not accessible enough and the existing impediments to experimenting and prototyping are not helping.

Cross-Industry Trends

These trends include:

  • The merging of intents and entities
  • Contextual entities. Hence entities sans a finite list and which is detected by their context within a user utterance.
  • Deprecation of the State Machine. Or at least, towards a more conversational like interface.
  • Complex entities; introducing entities with properties, groups, roles etc.
Chatbot Growth In Capability

There are both horizontal and vertical growth with chatbot technology. From the diagram above it is clear where this growth is taking place:

Vertical — Technology

The Conversational UI is moving away from a structured preset menu and keyword driven interface. With movement towards unstructured natural language input and longer conversational input.

Horizontal — User Experience

In this dimension the bot is transforming from a messaging bot to a truly conversational interface. Away from click navigation to eventual unrestricted compound natural language.

The Digital Employee

The end-game is where the digital employee, emerging from the chatbot environment, has evolved into areas of text and speech.

With contextual awareness on four levels:

  • Within the Current Conversation
  • From Previous Conversations
  • From CRM & Other Customer/User Related Data Sources
  • Across different mediums

The digital employee with grow across different mediums and modalities. Mastering languages with detection, translation, tone, sentiment and automatically categorizing conversations.

The Growth Of A Digital Employee Within An Organization

Mediums will include devices like Google Home, Amazon Echo, traditional IVR and more. As we as humans can converse in text or voice; similarly the digital employee will be able to converse in text or voice.

Chatbot Offerings Rating Matrix

In rating the eight chatbot solutions I looked at nine key points. Obviously NLU capability is key in terms of intents and entities. I was especially harsh on the extend to which entities can be applied in a compound fashion, annotated and detected contextually.

Dialog flow (state) development and management are also a key points; ease of development is important and to what extend collaboration is possible.

The other elements are self explanatory.

Key to Ratings

For different organizations, disparate element are important and will guide their thinking and eventually determine their judgement. For instance, even-though Lex does not feature in many respects, if a company is steeped in AWS for other service, Lex might be the right choice.

The sames goes for Oracle.

Chatbot Rating Matrix

Graphic Call Flow / Dialog Development Tools

For larger organisations and bigger teams, collaboration is important. Ease of sharing portions of the dialog and co-creating is paramount. Hence organizations have a need for graphic development environments.

Other teams prefer a native code approach.

Rating of GUI Form Call Flow Development & Editing

You will see that Microsoft has a fast growing and expanding bot framework environment with tools to match. What is attractive regarding Microsoft, is the native code approach within the Bot Framework coupled with the GUI of Composer. All underpinned by the exceptional power of LUIS.

Rasa with their tool called Rasa-X is so unique that it is hard to accurately categorize with the other environments. Rasa-X is graphic, it allows for editing and development, but is far more comprehensive.

NLU

Natural Language Understanding underpins the capabilities of the chatbot. Without entity detection and intent recognition all efforts to understand the user come to naught.

On some elements of a chatbot environment, improvisation can go a long way. This is not the case with NLU. LUIS has exceptional entity categorization and functionality. IBM Watson Assistant can also be counted as one of the leaders, with RASA.

Natural Language Understanding Capability

I also looked at the the integration of the NLU components into the other chatbot components. This is where Microsoft excels with their growing chatbot real-estate.

Scalability

Maturity of any framework is tested in an enterprise environment where implementations with diverse use-cases and ever expanding scale are present.

Scalability & Enterprise Readiness

Enterprise readiness is an evaluation criteria which does not enjoy the attention it deserves. Once vulnerabilities are detected, too much money and time have already been invested in the technology.

Overall Ratings

It is impossible to compare frameworks on a one-to-one basis, hence I created the five points of consideration as seen in the image below. It must be noted that one or more of these five elements, might be of higher importance to some organization than others. Hence that may draw them into a certain direction.

Again, if a company is already heavily invested in Oracle Cloud or AWS, then that will be a huge deciding factor for them. Overriding other considerations and easing the pain of other shortcomings.

Scoring Matrix Based On 5 Elements

Cost plays a big role, and this again speaks to the accessibility of environments like Cisco MindMeld and RASA; especially for initial prototyping.

Conclusion

This is a mere overview based on a matrix with points of assessment I personally deem as important.

And again, based in how important a particular point on the matrix is to you or your organization, will influence our judgement.

In the final analysis the software is to serve a purpose in your organization and current cloud landscape. The offering best suited for that purpose is the best choice for you.

Strengths & Weaknesses Of The Winners

--

--