Photo by JuniperPhoton on Unsplash

What Are List Entities & How Can It Improve Your Chatbot?

Extract Fixed, Closed Sets of Related Words With Synonyms

Introduction

There are a number of entity types which are available across chatbot frameworks, one of these are lists.

Lists are less glamorous than ML entities, entities with roles or composed entities. However, lists serves a specific purpose and there will most probably always be an implementation use-case for it.

Lists are used when you are looking for specific entities within an user utterance. Ideal scenarios are when you have a finite list of items, which might include:

• Towns / Cities
• Postal Codes
• Products / Services
• Employees etc.

Normalization adds consistency to the result yielded… For instance entries like john, Johnny, John, Jonh can be all normalized to Jonathan. These are referred to as synonyms.

List of items and their synonyms extracted with exact text match.

Build A Chatbot with Microsoft Bot Framework Composer & Composite Entities

List Entities In LUIS

List entities can be seen as a grouping of words which are:

• Fixed
• Closed
• Related

Here is no discovery involved through which new values are added.

Use the Recommend feature to see suggestions for new words based on the current list. If there is more than one list entity with the same value, each entity is returned in the endpoint query.

Lists are not machine learned; the defined and exact text is extracted when matched. The match is returned as an entity result.

List entities work well when you are dealing with text data which is:

• A known set.
• Is not dynamic in nature. If the list must self-expand simple entities which are contextually defined will work better.
• Fuzzy matching, stemming, plurals, and other variations are not resolved with a list entity.

Next, let’s look at a practical example of implementing a list using the following tools:

• LUIS
• Microsoft Bot Framework Composer
• Microsoft Bot Framework Emulator

How To Use Microsoft LUIS With Bot Framework Composer

Composer Setup

We can start this simple prototype with defining a new trigger which we call Countries.

Adding a new trigger in Bot Framework Composer

The next step is to create a Trigger phrase or intent, in this case the intent’s name will be #Countries. Here are the phrases which will set this trigger off.

Trigger phrases listed

- I want to check a country
- I need information on a country
- Country information

The basic dialog flow is the bot introducing the conversation topic, asking for input from the user and then doing the lookup to the list entity to return a country name. The country name is in essence a normalized value based on the synonyms detected.

Simple flow of the Countries trigger

The user input tab has all the defined values we need to capture the user input. The tricky part is defining the entity with the correct syntax.

The user input tab

Here is the full entity syntax for the list entity…The idea is that when a user enters a country code, population figure etc., it will normalize to the countries name.

- # list Entity country
@ list country =
- Afghanistan:
 -93
 -AF
 - AFG
 - 29121286
 - 647500
 - 20.65 Billion
- Albania:
 - 355
 - AL
 - ALB
 - 2986952
 - 28748,12.8 Billion
- Algeria:
 - 213
 - DZ,DZA
 - 34586184
 - 2381740
 - 215.7 Billion
- American Samoa:
 - 1-684
 - AS
 - ASM
 - 57881
 - 199
 - 462.2 Million
- Andorra:
 - 376
 - AD
 - AND
 - 84000
 - 468
 - 4.8 Billion
- Angola:
 - 244
 - AO
 - AGO
 - 13068161
 - 1246700
 - 124 Billion
- Anguilla:
 - 1-264,AI,AIA,13254,102,175.4 Million
- Antarctica:
 - 672
 - AQ
 - ATA
 - 0
 - 14000000
 - 0 Million

When you are satisfied with the changes, restart the bot and test.

Restarting and testing the bot

Ensure the bot is connected to a LUIS cloud instance.

An Overview Of Microsoft LUIS Machine Learned Entities

Testing The Dialog

When you have successfully deployed, you can enter one of the key phrases defined in the intent section to invoke or trigger this dialog.

When I enter a population, it matches that of American Samoa and the return prompt from the bot can be compiled with this dynamic value.

Testing your chatbot

Below is the JSON returned by LUIS, which is visible within the emulator.

{
  "recognizerResult": {
    "alteredText": null,
    "entities": {
      "country": [
        [
          "American Samoa"
        ]
      ]
    },
    "intents": {
      "TextInput_Response_USYSvV": {
        "score": 0.199661851
      }
    },
    "text": "I am thinking of a country with a population of  57881"
  }
}

Here is the full view of the emulator. The conversation is on the left, with the logs bottom right. The active links can be used to retrieve more information.

Microsoft Bot Framework Emulator

Conclusion

There is not a single entity which solves all conversational challenges…How entities are implemented also differ according to chatbot platforms and environments.

Careful consideration is required when deciding which entity types are assigned to various nouns or pieces of information which needs to be captured.

As we have seen here, List entities is a very simplistic and rudimentary entity type; but has a clear use-case and implementation scenario.

Read More Here:

Cobus Greyling

Cobus Greyling - Medium

.lu file format - Bot Service

Install Bot Framework Composer - Bot Composer

microsoft/BotFramework-Emulator