IBM Watson Assistant Now Has Improved Intent Recognition
Improve Your Chatbot’s Ability To Understand Customers With A New Intent Detection Model
Introduction
IBM Watson Assistant is introducing new functionality at a remarkable cadence. Notably these changes are not disruptive to existing features and augment or compliment existing functionality.
The previous big announcement from Watson Assistant was Actions…prior to the enhanced intent detection model.
November 2020 IBM Watson Assistant released a beta version of their new intent detection model. Intents are the frontline of any conversational interface.
Intents are categories of what users’ intentions are; really the frontline of any chatbot. Accurately classifying the user utterance according to an intent is of utmost importance.
Various parameters can be set in each dialog node to determine relevance, and the most popular measure here is intent scores.
What Is New?
According to IBM the new intent detection model is faster and more accurate. It combines:
- Traditional Machine Learning
- Transfer learning, and
- Deep Learning Techniques.
Less training data is required, which is convenient for prototyping.
The new model is available as a beta feature for English dialog and actions skills only.
Once the new feature is activated, Watson Assistant needs to perform training.
Enhanced detection can improve your assistant’s ability to classify what customers say with the right intent, more often.
Re-test your intents after applying the enhanced version. Changing versions will automatically retrain the skill, which might cause slight differences in results. You may keep or switch back to the classic version.
Enabling The Enhanced Intent Detection Model
Firstly, what I find interesting is that the new model is used by Action skills by default and cannot be disabled. This most probably is an indication if what the future holds.
In order to enable the new model in a dialog skill:
- From the Skills page, open your skill.
- From the skill menu, click Options.
- On the Intent detection page, choose Enhanced version.
Test the new intent detection capabilities in the “Try it out” pane.
Testing The Result
One thing to note is the ability to switch back to the classic version should the results yielded not be satisfactory.
Alternatively, if left on, rigorous test is required. One of a few reasons for this is that many dialog nodes have conditions set based not only on the intent detected, bot the score of the intent.
These scores are used as thresholds to determine which node to invoke.
I looked at two scenarios:
- A non-ambiguous short user utterance.
- An ambiguous user utterance.
For the utterance: “What are the hours?” The intent score for the correct intent is 0.02 higher. The two next best intents are also marginally lower.
This simple tests shows a general improvement in the model.
The second utterance tested was “Thanks you really were a great help.” Which is an awkward sentence, and ambiguous looking at the list of intents available.
Seemingly the new model detected the ambiguate by bringing down the score of the correct intent; #Thanks. The intent #Help moved up.
Conclusion
With the promise of less training data and improved accuracy, the new model is compelling.
A drawback is the lack of insight or understanding of what is happening under the hood. For others this might be an advantage.
The fact that IBM Watson Assistant is evolving at this pace bode well for organizations having invested in the technology.
