Nuance Mix, Conversational AI Tooling & Conversation Abstraction
And How To Orchestrate Customer Service Experiences
Introduction
All conversational AI frameworks need to deal with the same problem…
And this problem is, creating an ecosystem to craft natural conversations.
However, to build everyday, natural conversations with any given tool, all the concepts, objects and attributes of a real-world conversation need to be abstracted.
Hence the process of creating abstract concept-objects by mirroring common features or attributes of a non-abstract real-life conversation.
These abstracted elements in turn need to be orchestrated into a development environment to create a Conversational AI framework.
In this process of abstraction, detail are lost, through generalisation.
- Generalisation is necessary in order to constitute building blocks…
- Generalisation is also important to be accurate in crafting conversations, and have control over the conversations. We want control, with flexibility…
As seen above, the abstracted conversational elements of the Nuance Mix Conversational AI framework. More detail on this later in the story…
Control and flexibility; these two needs to be in balance:
- Control without Flexibility negates key conversational elements like digression, disambiguation, clarification, domain/relevance, compound intents & entities.
- Flexibility without Control definitely has a certain allure, but this gives rise to the lack of affordances for fine-tuning conversations. This is very much the case with OpenAI’s Language API. Immense flexibility which is enticing, but limited fine-tuning.
And lastly, the abstracted elements must then be used to constitute conversations which simulate real-world conversations as close as possible. Hence closing the loop.
Nuance & Microsoft
On 4 March 2022 Microsoft made the announcement that the acquisition of Nuance has reached completion. Nuance has been seen by many as having missed the boat on Conversational AI. Commentators saw Nuance as marginalised as the veteran in an industry they should have dominated.
But this changed due to two reasons:
- The Microsoft acquisition
- Nuance Mix
February 2020 saw the beta release of Nuance Mix, and looking at their release notes, there has been a steady cadence of 2 to 3 releases per month.
This level of activity and development speaks of continued value delivery and investment into an already astounding product.
Dialog Design Affordances
The abstraction of conversational elements are becoming more astute. This is evident in the approach of Cognigy, Core AI, a platform like HumanFirst and Nuance Mix.
There are for sure other frameworks following suit, which I have not yet gotten to…
- Abstraction needs to be performed in such a way that teams can still collaborate on the same conversational assistant.
- No-code to lo-code is the primary approach.
- Design and Development are fusing.
- There is significant focus on upskilling and training. This is evident in Nuance Mix’s top tier of elements; Learn & Manage.
- Another focus is continuous analysis, and assistant iteration.
- This orchestration of abstracted elements might seem complex at first, this is for sure the case with Mix. But once familiar with the environment, the development environment is intuitive.
- The initial avenue seems to have been strengthening the four pillars of chatbots (Intents, Entities, Dialog State Management, Messages) or even deprecating one of these pillars.
- But, the current approach is merging these pillars, and cross-imbedding functionality. Adding settings and option to the NLU portion, adding more structure to NLU in the form of nested intents and entities.
- But also assigning specific portions of the flow to specific portions of the NLU.
- Mix does a good job of also extracting the messaging portion of the dialog, and allowing settings in that area. As seen below.
In broad terms there are four dialog design and management approaches…
- A pro-code approach where the call-flow is defined by lines of code, this is very much the approach of the Microsoft Bot Framework, Cisco MindMeld and in AWS with LEX using Lambda and one could argue to some extend Rasa.
- A conversation state table approach, this we have seen in Microsoft Power Virtual Agents, IBM Watson Assistant Action Skills. And then the flagship Watson Assistant Dialog Skills conversation state development interface.
- Machine Learning stories is really the domain of Rasa, but this approach has also been attempted by AWS via Alexa Conversations.
- The last and most broadly implemented dialog design and management approach is a design canvas. This is a canvas where dialog design elements, flow nodes (there are multiple terms) are dragged, dropped and connected; with property settings. This approach is used by Google’s Dialogflow CX, Kore.ai, Cognigy and others. And part of this list is of course Nuance Mix.
Where a platform like Cognigy have multiple dialog design elements (or flow nodes as they call it), Mix has only 14 node types. Where Cognigy has many more dialog nodes, their approach is more horizontal. More on this in the next section…
Nuance Mix & Dialog Design Elements
Nuance Mix only has 14 dialog design elements:
Start, Question & Answer, Message, Decision, Data Access, Question Router, Intent Mapper, SMS (not yet available), Email (not yet available), Component call, Enter, External Actions, Transfer and End.
But, these dialog design elements have an array of settings and options. This stacking or segmenting the options allow for quick initial familiarity and onboarding and levels to drill down for enhanced fine-tuning.
As seen above, the dialog development and management area can be divided into four areas.
- Red: The first being the dialog node elements.
- Yellow: The second the design canvas.
- Green: Thirdly properties menu.
- Purple: The settings for each op the option settings.
Conversation Abstraction
Nuance Mix addresses quite a few pain points very accurately…
- The user interface’s top layer gives a user the option to go to training material or manager bots. The training material is well organised and complete.
- Under projects six headings are available to manage different projects. These include user access, Domains, Import and Export and more.
- From here there is a clear separation between developing and managing NLU and the Dialog. NLU does make a return later in the dialog set of elements, for obvious reasons.
- NLU development and management are divided into development, optimisation and discovery.
- The Mix Dialog Design and development canvas is rigid and non-responsive in the sense that dialog elements cannot be dragged, dropped or moved around. This can be seen as an advantage, where inadvertent dialog flow changes are negated, and the supporting elements can be used instead.
- Mix manages intents well and digression is quite straight forward to achieve.
- Different components premised on various intents can be created, and confirmation can be added prior to digression.
- The NLU environment is strong in terms of testing, adding new utterances, quick and detailed JSON document feedback. See below…
- Nuance Mix does support on-premise and self-hosted deployments of the Text To Speech (Speech Synthesis), Speech To Text (Automatic Speech Recognition), NLU and dialog/Dialog Orchestration runtime engines.
An extract of the rich JSON document with detail on intents, entities and more.
Conclusion
In the coming days I will be doing more prototyping on Nuance Mix. Suffice to say, currently the interface feels a bit lean with limited affordances. This will truly be tested by complex implementations which will need to scale.
A few considerations on Nuance Mix’s strongpoints and a few possible areas of improvement…
The Good
- Nuance Mix Outline is coming soon. It is described as a process of creating multiple conversational paths with the ease of writing transcripts. These dialogs will be converted into a dialog tree or conversation. This is a potential game changer and a much needed differentiator for Nuance Mix. This could be a mix between Rasa ML stories and a dialog design canvas approach.
- Dialogs or flow messages to the user can be managed separately, this reminds much of Microsoft’s Composer dialog management.
- Validation can be performed on the call flow, with Warnings & Errors. This can safe much time in debugging and troubleshooting. As seen below…
- Channel views are available for testing, hence Nuance Mix can be used to combine different modalities into one design or voice application.
- Different components can be created. These components can be seen as segments of the conversation. Components can be linked via intents, and the conversation can digress from one component to the other.
- As with Cognigy, as the application is tested out via the test pane, the position or current state of the conversation is displayed within the dialog flow. Hence the conversation is tracked and the current state displayed in the designed flow.
- The selection of available nodes might seem limited. But each node type has extensive settings. Hence having the approach of surfacing simplicity while having the complexity available under the hood if needed.
A Few Considerations
- The design and development canvas is not interactive, the menus and settings on the side need to be used. It does feel a bit debilitating to not be able to drag, drop and manipulate nodes in the canvas my moving them.
- The grey user interface is probably not ideal.
- Nuance’s strong points are definitely multi-modality, STT, TTS, speech enablement in general etc. There is definitely room for improvement with regards to the dialog tree interface.
With the delivery of Nuance Mix Outline, Mix might step into a new dimension of conversation building.
