Build A Chatbot Conversational App with Haystack & HuggingFace
ChatGPT and HuggingChat are both web-based Conversational Apps consisting of a UI, conversational memory and access to a LLM. Here I show how you can create your own conversational app using open-source technology.
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
In the coming week I aim to compile a matrix with the most notable LLM-based development frameworks for conversational applications.
This article covers a simple notebook example on how to build a conversational app which has memory. There are two main components used for this demo, HuggingFace and Haystack.
Via Hugging Face, hosted Inference APIs can be used to access Large Language Models using simple HTTP requests.
You don’t need to download models, perform any fine-tuning or training. All you need is an API Key from HuggingFace, as seen below:
Haystack is an open-source, pro-code framework to build Autonomous Agents, prompt pipelines, search tools and more.
The conversational app demo shown below will make use of three nodes; PromptNode, ConversationalAgent & ConversationSummaryMemory.
PromptNode
The PromptNode is initialised with three parameters, model_name, api_key, and max_length to manage the model output.
from haystack.nodes import PromptNode
model_name = "OpenAssistant/oasst-sft-1-pythia-12b"
prompt_node = PromptNode(model_name, api_key=model_api_key, max_length=256)Here is the simplest implementation of the PromptNode:
pip install --upgrade pip
pip install farm-haystack[colab]
from haystack.nodes import PromptNode
prompt_node = PromptNode()
prompt_node("What is the capital of Germany?")And the output:
['berlin']ConversationSummaryMemory
Conversation memory is important for conversational apps to have a human-like element.
Follow-up questions can be asked which reference previous conversational context in an implicit fashion.
ConversationSummaryMemory is used to save space and also LLM tokens.
The summary has a brief overview of the conversation history and will be updated as the conversation continues.
Implementing Conversation Summary Memory:
from haystack.agents.memory import ConversationSummaryMemory
summary_memory = ConversationSummaryMemory(prompt_node)ConversationalAgent
And lastly, the conversational agent:
from haystack.agents.conversational import ConversationalAgent
conversational_agent = ConversationalAgent(prompt_node=prompt_node, memory=summary_memory)A conversational agent is an agent holding conversational memory. To read more about agents, you can refer to this article.
Here is the complete code for the Conversational Agent:
pip install --upgrade pip
pip install farm-haystack[colab]
from getpass import getpass
model_api_key = getpass("Enter model provider API key:")
from haystack.nodes import PromptNode
model_name = "OpenAssistant/oasst-sft-1-pythia-12b"
prompt_node = PromptNode(model_name, api_key=model_api_key, max_length=256)
from haystack.agents.memory import ConversationSummaryMemory
summary_memory = ConversationSummaryMemory(prompt_node)
from haystack.agents.conversational import ConversationalAgent
conversational_agent = ConversationalAgent(prompt_node=prompt_node, memory=summary_memory)
conversational_agent.run("What are the five biggest countries in Africa?")
conversational_agent.run("What is the main languages spoken in these countries?")
conversational_agent.run("Are any of the counries land-locked?")
print(conversational_agent.memory.load())⭐️ Please follow me on LinkedIn for updates on LLMs ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
