Chatbot Demo: Smart Entity Extraction
Glean multiple entities from a complex dialog
When testing a chatbot interface, the plan which is most often followed, is to stick to the dialog or script, and enter short dialogs with single intents. Typically the “happy path” which should work well. This is a scenario in conversation where dialogs are shorter in length and with single intents and or entities.
However, once a user veer off script, and the dialog become longer with multiple sentences and crowded with intents, the bot struggles to extract the intents and entities.
In this case, a the stronger weighted intent is favored and the others discarded. Or, the chatbot responds with an “I don’t understand” dialog and most often the dialog does not progress beyond this point.
There are cases where users enter text with up to 4 intents in one dialog; or more. Also, the split between intents and entities are not always clear.
This problem necessitates the use of a bona fide NLP (Natural Language Processing) environment. Or at least a higher order NLP first pass for dialogs. Any solution making use of keyword spotting or fixed phrase recognition will fall short here and be caught out as a NLP/NLU impostor.
The solution…
The ideal is for the NLP system to parse the user dialog, and then in turn detect multiple intents within the dialog and capture the entities of that specific intent. This leads to a merger of intents and entities being contextualized.
Hence entities are recognized within a given intent context. This is the key ingredient, the chatbot being able to detect the context of the user utterance. Also, the relative context, within the dialog, of valuable intents and/or entities within the utterance.
As per the example above, the items of interest is picked out of the customer dialog. IBM Watson Assistant makes this particularly easy to setup and configure.
For an in-depth look at the IBM Watson Assistant’s approach to contextually parse user input, have a look at this article I wrote on the subject.
More reading and general information on the subject…
IBM Watson Assistant Contextual Intents:
https://medium.com/@CobusGreyling/contextual-chatbots-e7ab04c03d1f
Natural Language Processing:
https://en.wikipedia.org/wiki/Natural_language_processing
Natural Language Understanding:
https://en.wikipedia.org/wiki/Natural-language_understanding
Unstructured Data:
https://en.wikipedia.org/wiki/Unstructured_data
