What Is GODEL (Large-Scale Pre-Training for Goal-Directed Dialog)?
On May 2022 Microsoft announced GODEL. GODEL is designed for general-domain conversation and is fully open-sourced.
We have seen generation used in creating a chatbot via Large Language Models.
One area where generation is being considered, is with creation and management of dialog/flow design, development and management. The notion of creating a dialog flow from example conversations, and having a machine learning approach to it. Where the systems determine the next dialog to present to the user, based on probability.
In 2019 I considered the approach of Rasa, which they call Interactive Learning. With Interactive Learning you can write dialogs (machine learning stories). This is where you write your stories while you are talking to your bot. And while you talk to your bot, the dialog flow is mapped in a browser.
Kore AI also has an innovative approach to building dialogs with their Conversation Driven Dialog Builder, with a conversational first approach. This does remind somewhat of Rasa’s Interactive Learning and Conversation Visualisation.
According to Kore.ai, conversation-driven Dialog builder automatically converts the storyboard scenes into a Dialog Task. Designers can focus on visualising the end-user conversation before building the dialog.
As seen below in the screen extract from Nuance Mix, another case in point. This imminent feature of Nuance Mix allows for reverse engineering a dialog flow from a written transcript.
Considering The GODEL Architecture
The pre-trained model of GODEL can be fine-tuned and adapted to be applied to new dialog tasks.
The aim of GODEL is to solve for the long-standing impediment to general-purpose open-ended conversational models. These conversational models can be task-oriented or merely open-ended, hence non-domain specific small-talk.
GODEL is striving to deliver human-like conversations which attain a high level of utility. The model should also have the ability to generate responses based not just on the context of the conversation, but also on external information, content that was not part of the dataset when the model was trained.
I think a key takeaway of the GODEL architecture should be the following…
Consider the JSON format in which the training data needs to be:
"Context": "Please remind me of calling to Jessie at 2PM.",
"Knowledge": "reminder_contact_name is Jessie, reminder_time is 2PM",
"Response": "Sure, set the reminder: call to Jesse at 2PM"
The fields catered for are:
Context — The context of the current conversation, from start to current turn.
Knowledge — External or environment state represented in plain text.
Response — The virtual agent response message or text. This can be a template, an api call or natural language generation.
The three elements mentioned above, are vital to a well managed conversational experience.
The context of the conversation needs to be maintained, or at least a brief overview of the conversational session.
Knowledge, this could be entities or slots captured.
Response should also be contextual, but the hard part is text generation. Generating a response which is contextual, linguistically coherent and correct. And with the correct information imbedded, gleaned from knowledge.
Cobus Greyling - City of Johannesburg, Gauteng, South Africa | Professional Profile | LinkedIn
Rasa Hero. NLP/NLU, Chatbots, Voice, Conversational UI/UX, CX Designer, Developer, Ubiquitous User Interfaces…
Cobus Greyling - Medium
Read writing from Cobus Greyling on Medium. NLP/NLU, Chatbots, Voice, Conversational UI/UX, CX Designer, Developer…
Eliza Language Technology Community - Language Technology: Conversational AI, NLP/NLP, CCAI…
ELIZA - Where language technology enthusiasts unite.
GODEL: Combining goal-oriented dialog with real-world conversations - Microsoft Research
They make restaurant recommendations, help us pay bills, and remind us of appointments. Many people have come to rely…
GitHub - microsoft/GODEL: Large-scale pretrained models for goal-directed dialog
This repository showcases building goal-directed dialog using GODEL, and contains the dataset, source code and…