Current Approaches To Chatbot QnA & The Microsoft Approach
This article covers the existing Chatbot QnA methods & how Microsoft is approaching answering questions.
The TL;DR
- Microsoft is deprecating Azure QnA Maker on 31 March 2025.
- The two alternatives offered by Microsoft are answering questions using a knowledge base…
- or answering a question by searching a body of text.
- Knowledge bases.
- Another method is existing technology used in LLMs and 🤗HuggingFace Pipelines and commonly used within LLM generative models.
- Quick Reply Intents is a feature pioneered by Oracle Digital Assistant.
- Knowledge Intensive NLP (KI-NLP) is being pioneered by LLMs and Meta’s Sphere.
- Search / Wikipedia is the most common avenue for answering primarily non-domain specific QnA.
Please follow me on LinkedIn for updates on Conversational AI Research. 🙂
What Is The Microsoft Approach?
In the past my general feeling was that Microsoft’s services were too disparate and the Azure QnA Maker is a good case in point. In the past I found it was vulnerable to understeering and oversteering in terms of varying content.
The new strategy from Microsoft…
Microsoft has 13 categories of language technologies in their language studio…I believe this speaks to the Microsoft approach of having separate but distinctly defined “model cards” with their langauge studio.
API’s created via these models can then be orchestrated into skills or a digital agent, and more.
Any of these language studio elements listed above, can be used in isolation for human langauge related tasks, or orchestrated into a voice or chat based digital assistant.
Please follow me on LinkedIn for updates on Conversational AI Research. 🙂
Two Approaches to Question Answering
The image below shows the two API tiles to access language studio’s QnA functionality:
- Answering questions
- Custom question answering
Answer Questions (AQ)
Answer questions reminds me of the approach taken for question answering using a LLM generative model.
- This approach and functionality are commonly available in LLMs.
- It is well suited for highly dynamic and varying bodies of text to search.
- The text to be searched cannot be too long or complex and will most probably be constituted by customer conversations or interaction data which is retrieved and in turn searched.
- This approach can also be used where an intent-driven process extracts a body of information or a document, and Answer questions is used to drill down further for a specific answer.
- Most probably OpenAI’s GPT3 model is used under the hood for this service.
Custom Question Answering (CQA)
Custom question answering is a more structured approach to create a knowledge base API. Documents and URL’s can be used and a basic dialog and follow-up facts can be defined, as shown later in the article.
- CQA projects can be created using URL’s or Files.
- It can also be used for Chitchat.
- Both AQ and CQA have language restrictions.
- As is the case with Chitchat, which is currently available in the following languages: Chinese Simplified, English, French, German, Italian, Japanese, Korean, Portuguese, and Spanish.
- Chitchat has five persona settings, Professional, Friendly, Witty, Caring, Enthusiastic.
- The CQA has a chatbot like test pane on the right, and responses can be set to short, long, or both. This is useful when response lengths will vary based on the data searched.
- Custom questions and responses can be added with follow-up prompts.
- Synonyms can be added which might be used in queries.
- Continuous improvement is available where users can review, accept, or reject suggested alternate phrases for questions.
- These suggestions are gleaned from user interaction data.
Answer Questions — The Detail
The Answer questions functionality reminds me quite a lot of a LLM generative model approach. Below is a LLM example from BLOOM via 🤗HuggingFace.
The same body of data (training data) was used in BLOOM and Microsoft’s Answering questions functionality, in order to compare the two.
The basic approach is to define a piece of information, and tag it as Facts, and then tag the Question, as seen below. The “Answer:” text is defined and the generative text model generates the answer to the question where prompted.
Facts: "The Space Needle is one of the most recognizable landmarks in the world and is a treasured Seattle icon. Built for the 1962 World’s Fair—the Century 21 Exposition whose theme was “The Age of Space”—the tower’s futuristic design was inspired by the idea that the fair needed a structure to symbolize humanity’s Space Age aspirations. Since its grand opening on April 21, 1962, the landmark continues to symbolize the innovative and forward-thinking spirit of Seattle. Located at Seattle Center, the Space Needle stands at 605’ tall and is one of the most photographed structures in the world. The tower’s 520’ saucer-shaped “top house” offers visitors Seattle’s only 360-degree indoor and outdoor panoramic views of downtown, Mount Rainier, Puget Sound, and the Cascades and Olympic mountain ranges. To further elevate the visitor experience, the Space Needle embarked on a multi-year venture focused on the preservation and renovation of the 57-year-old icon. The first phase of the renovation, completed in late summer 2018, includes breathtaking, multi-level, floor-to-ceiling glass viewing experiences including an upper level outdoor observation deck with open-air glass walls and Skyriser glass benches. The Oculus, a grand wood, steel, and glass staircase connects the upper-level to The Loupe, the world’s first, and only, rotating glass floor below. The new glass floor experience opened up a never-before-seen downward view of the structure and the surrounding area, putting Seattle, literally, at your feet. Visit the renovation page to learn more about how the Space Needle is further enhancing its visitor experience. The Space Needle is proud to support local organizations who make a positive difference in the Puget Sound region through its community giving and engagement programs. Visit the Space Needle Foundation page to learn more about the Space Needle’s community giving and guidelines."
Question: "What is oculus?"
Answer:The short answer BLOOM returned:
"The Oculus is the name given to a grand staircase and observation deck.So lets compare the BLOOM example we just saw, with the Microsoft Answer questions functionality, using the same data…
Below again is the same data, but now which is defined within the Microsoft console. Followed by the question, and the generated answer.
Here is the JSON result:
{
"answers": [
{
"answer": "The Oculus, a grand wood, steel, and glass staircase connects the upper-level to The Loupe, the world’s first, and only, rotating glass floor below. The new glass floor experience opened up a never-before-seen downward view of the structure and the surrounding area, putting Seattle, literally, at your feet. Visit the renovation page to learn more about how the Space Needle is further enhancing its visitor experience.",
"confidenceScore": 0.9820879101753235,
"id": "id__1417",
"answerSpan": {
"text": "a grand wood, steel, and glass staircase",
"confidenceScore": 0.56610686,
"offset": 11,
"length": 41
},
"offset": 1218,
"length": 420
},
{
"answer": "The first phase of the renovation, completed in late summer 2018, includes breathtaking, multi-level, floor-to-ceiling glass viewing experiences including an upper level outdoor observation deck with open-air glass walls and Skyriser glass benches. The Oculus, a grand wood, steel, and glass staircase connects the upper-level to The Loupe, the world’s first, and only, rotating glass floor below. The new glass floor experience opened up a never-before-seen downward view of the structure and the surrounding area, putting Seattle, literally, at your feet.",
"confidenceScore": 0.9484376907348633,
"id": "id__1417",
"answerSpan": {
"text": "a grand wood, steel, and glass staircase",
"confidenceScore": 0.6114064,
"offset": 260,
"length": 41
},
"offset": 969,
"length": 557
},
{
"answer": "The Space Needle is one of the most recognizable landmarks in the world and is a treasured Seattle icon. Built for the 1962 World’s Fair—the Century 21 Exposition whose theme was “The Age of Space”—the tower’s futuristic design was inspired by the idea that the fair needed a structure to symbolize humanity’s Space Age aspirations. Since its grand opening on April 21, 1962, the landmark continues to symbolize the innovative and forward-thinking spirit of Seattle.",
"confidenceScore": 0.10796824097633362,
"id": "id__1417",
"answerSpan": {
"text": "Space Needle is one of the most recognizable landmarks in the world and is a treasured Seattle icon",
"confidenceScore": 0.17642552,
"offset": 3,
"length": 100
},
"offset": 0,
"length": 466
}
]
}Custom Question Answering — The Detail
In order to crate the knowledge base, different types of data can be uploaded. This data can be submitted via URL’s, or files can be uploaded. Chitchat, which is sometimes referred to as smalltalk, can also be defined. When uploading, the file structure can be defined or auto-detect can be utilised.
I used the URL to the HumanFirst website and an automatic process kicked-off to collect data. As seen below, there is a test pane on the right with a few options. Responses can be toggled between long answers and short answers with advanced options available.
Fine-tuning of the Knowledge Base can be defined with question and answer pairs as seen below…
Here you see the question and answer pairs after being defined…
And finally, the test on the right with the desired result.
In Closing
The approach from Microsoft does seem to leverage a LLM in general, and most probably OpenAI’s GPT3…
The quick generative approach of Answer Questions is really innovative and a great example of how a LLM can be leveraged for real-world implementations.
It is interesting that QnA development environments are becoming more feature rich and in some instances are starting to emulate chatbot development frameworks.
Please follow me on LinkedIn for updates on Conversational AI Research. 🙂
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
