Photo by Maxim Potkin on Unsplash

Migrating To Rasa Open Source NLU From Microsoft LUIS

I migrated A Microsoft LUIS NLU Application To Rasa NLU and this is what happened…

Introduction

Many Chatbot Designs Are Dependent On A Standalone NLU API

Migrating your chatbot/NLU solution from one environment to another is not something which will happen frequently. Should you consider it, the prime reason will most probably be cost, hosting options, customization.

The good news:

• Most NLU environments export into a text/JSON format which can be easily manipulated to suit the new required format.

Microsoft Bot Architecture overlaid on the Rasa architecture.

• Some architectures have a clear separation between NLU and the other components (State Management, dialog, integration, forms etc.)
• The Microsoft environment is a good example of this clear separation. Hence LUIS, can be migrated to Rasa NLU while the bot framework or composer environment remains largely intact. With larger implementations in can make sense to split the functionality across different platforms.
• Intents are usually very easily transformable.

The not-so-good news:

• Various different entity types exist between vendors and environments. Migrating entities successfully sans human intervention is in most cases not possible. IBM Watson has very generic annotate-able entities. But LUIS has the most advanced and complex array of entity types.
• Thorough testing will be required to ensure no vulnerabilities are introduced in the process.
• If you do not have segmentation of chatbot elements (NLU, State Machine, abstraction/integration etc.) this process will be much harder. Think here IBM Watson Assistant.
• It is not a bad thing, but structures and types are being introduced to entities, which complicates migration. These structures differ considerably between the various platforms. Which means entities cannot merely be transferred. The best fit in the new environment needs to be identified and the structure transposed.

Rasa…
Here are a few reasons why we see developers switching:
Faster: Runs locally — no http requests and server round trips required

Customizable: Tune models and get higher accuracy with your data set

Open source: No risk of vendor lock-in — Rasa is under the Apache 2.0 license and you can use it in commercial projects.

Rasa NLU Only

Although valuable functionality will fall along the wayside, Rasa can be used very conveniently as a NLU Engine and API alone.

Using NLU Only

Intents are defined, with annotated entities. Hence entities are defined by their context.

The LUIS Environment

The LUIS Environment created for the migration is a simple travel information NLU configuration. Three of the four available entity types in LUIS were used:

• Machined Learned
• List
• Regex

Pattern.any is a very specialized entity type and I thought not to include it.

Entity types within LUIS

The three entities created are:

• ConfirmationCode: Entity of Type Regex expecting a 5 digit code constituted only by numbers.
• Continents: Entity of Type List.
• TravelDetail: Made up of a Machine Learned entity type.

Three Entities created covering each of the three types.

Below is a view of our single intent called Travel with the example utterances. You will see that these utterances are relatively complex with multiple entities per utterance.

Annotating Utterance Example with Entity Elements

We could break these up into multiple intents to make the intents simpler. However, LUIS allows us to create complex entities, thus simplifying the intents process.

We start by defining a single entity named:

• Travel Detail

Within this entity, we defined three sub-entities. You can think of this as nested entities or sub-types. The three sub-types defined are:

• Time Frame
• Mode
• City

From here, we have a sub-sub-type for City:

• From City
• To City

Adding Sub-Entities: ML Entity Composed of Smaller Sub-Entities

Here are the intent examples, used to train the model with the entity, sub-types, and sub-sub-types; fully contextualized.

Annotated Intent Examples

Testing the interface in the test console is demonstrated here…

Testing the NLU Application

Lastly the LUIS NLU Application is exported in a JSON format.

Export the NLU application in JSON format

Rasa NLU Process

The Rasa procedure is detailed on this page of the Rasa website:

Rasa as open source alternative to Microsoft LUIS - Migration Guide

During the import process, this error message is displayed multiple times…more about this later.

UserWarning: Your luis data schema version 7.0.0 is higher than 5.x.x. Training may not be performed correctly.

Testing The Rasa NLU

The same three utterances were used after migration to Rasa NLU…

• All intents were detected correctly.
• Regex and List entities were not detected.
• Machine Learned entities where detected very accurately. However, the nested entity types were not detected. But this did not surprise me.

Rasa NLU detected all the entities and intent for the machined learned entities.

Conclusion

The LUIS environment has changed dramatically over the last few months, Machine Learned Entities were introduced to LUIS in November of 2019. Hence I am not surprised that Rasa is two LUIS schema versions behind.

Will I use the auto migration tool? It might be a useful starting point.

However, the approach I will follow is:

• Export the LUIS NLU Application in JSON format.
• Plan & Decide which Rasa entity types best match and represent the LUIS entity types.
• Create a Rasa nlu.md file based on the LUIS JSON file. This could be manual or programmatic.

I am sure Rasa’s entity structure will also evolve over time; as they are also introducing entity types; currently in Beta. Hence your NLU entity situation will not be static and evolve over time as your NLU environment evolve. Hence continuous and ongoing consideration is necessary to improve the NLU performance and leverage new functionality.

Read More Here:

Cobus Greyling

Cobus Greyling - Medium

Rasa as open source alternative to Microsoft LUIS - Migration Guide

LUIS (Language Understanding) - Cognitive Services - Microsoft