Jan 26
LangChain Is OpenSource Software For LLM Dialog State & Contextual Memory Management
Langchain is an open-source initiative which is addressing one of the pressing issues regarding Large Language Models (LLMs)…and that is managing a conversation.
Natively LLM APIs work well out-of-the-box for single dialog-turn, command-and-response type scenarios.
But what if you want to use a LLM for a multi-turn dialog between the user and the LLM? Hence the same functionality you expect to find in a chatbot or digital agent?
You have two options…
1️⃣ The first avenue is making use of a Conversational AI Development Framework like Cognigy, OneReach AI and a few others to integrate to LLMs.
However the premise of making use of it, will only be to have access to a dialog management environment. And with this comes additional overhead in the form of cost, unwanted functionality and complexity.
2️⃣ The second option is to write your own dialog management software.
💡 Hence LangChain makes a lot of sense for enabling LLMs for dialog management. LangChain is a thin pro-code layer which converts (sequential) successive LLM interactions into a natural conversational experience.
Be it managing the prompt engineering process, collecting data from the user in a conversational manner, API integration, dialog development, conversation context & memory, and more…
In the words of LangChain:
LLMs are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using these LLMs in isolation is often not enough to create a truly powerful app — the real power comes when you are able to combine them with other sources of computation or knowledge.
But First, What Are The Elements Of Dialog Management?
⚙️ Dialog State/Flow Management
This is the process of managing each conversation within a pre-defined decision tree. For each conversation the chatbot must know, based on available criteria, what the next dialog is to go to.
The dialog state is usually defined via a dialog flow editor. Dialog flow editors can be pro-code, low-code or no-code.
⚙️ Conversational Context and Memory
Conversational context makes the user’s conversational experience more natural and intuitive. Contextual awareness allows for the bot to make assumptions based on previous user utterances. It also helps with the bot not having to re-prompt users for data already entered.
⚙️ Long Term Conversational Memory
People like talking to the same customer care representative, due to the fact that there is long term memory. And this memory optimises the call and makes for a more efficient experience.
The same can be done via a chatbot.
⚙️ Integration to external APIs
Dialog Management includes functionality to make calls to an external API, process the data and include it as part of the narrative. Think of a weather bot, which would need to access a weather API to retrieve and present for a requested city.
⚙️ Knowledge Base Integration and QnA
QnA and Knowledge Base functionality is becoming more important and most conversation flow designers has, or make provision for some sort of knowledge base access.
⚙️ Development Interface
Generally there are four approaches to dialog flow development interfaces.
Of these, visual dialog flow editors are the most popular due to its visual and no-code nature. As seen below…
Considerations When Using LangChain
⏺ The LangChain application will have to be hosted somewhere. Hence with LangChain comes the overhead of hosting, processing, maintaining code, business continuity planning, redundancy, regional availability and latency.
⏺ LangChain is perfect for demonstration applications and high-fidelity prototyping. However, for a production implementation system architecture will be important.
⏺ LangChain is pro-code, hence there will be a divide between design and development. With developers having to interpret the design. In recent years we have seen the design process being leveraged. Hence the design process via a no-code GUI is exported into a production ready version of the flow.
⏺ As an application grows, complexity will also increase, deployments will be more hazardous and regression testing of the conversational interface will be important.
⏺ The hope is that much of the error handling, fall-back for out-of-domain etc can be offloaded to the LLM for management.
⏺ A big part of the LangChain development will be conversation management aspects.
Code Examples
The best way to get a grasp on LangChain is to consider a few prototypes…
The most basic example of using LangChain to query OpenAI:
pip install langchain
pip install langchain[all]
pip install openai
import os
os.environ["OPENAI_API_KEY"] = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
from langchain.llms import OpenAI
llm = OpenAI(temperature=0.9)
text = "What day of the week was it on 15 April 1973?"
print(llm(text))Adding a variable:
from langchain.prompts import PromptTemplate
prompt = PromptTemplate(
input_variables=["product"],
template="What is a good name for a company that makes {product}?",
print(prompt.format(product="typing machines"))
llm = OpenAI(temperature=0.9)
text = (prompt.format(product="typing machines"))
print(llm(text))⬇️ Notebook view:
The following prototype is a good example of how LangChain can be used to automate prompt engineering, and leverage LLMs for conversational context:
from langchain.llms import OpenAI
from langchain.chains import ConversationChain
from langchain.chains.conversation.memory import ConversationBufferMemory
llm = OpenAI(temperature=0)
conversation = ConversationChain(
llm=llm,
verbose=True,
memory=ConversationBufferMemory()
)
conversation.predict(input="Hi there!")Response:
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi there!
AI:
> Finished chain.
Hi there! It's nice to meet you. My name is AI. What's your name?Input:
conversation.predict(input="My name is Cobus")Response:
> Entering new ConversationChain chain...
Prompt after formatting:
The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Current conversation:
Human: Hi there!
AI: Hi there! It's nice to meet you. My name is AI. What's your name?
Human: My name is Cobus
AI:
> Finished chain.
Nice to meet you, Cobus! What can I do for you today?This approach is not optimal for longer conversations as the prompt size sent to the LLM will just grow too big.
In this final example, the user is prompted for input and this is used to generate the response, it also illustrates how two chains are combined.
title = input('What is your title? ')
print ('I have your title as ' + title)
era = input('Lasty, from what era should the text be? ')
print ('I have your era as ' + era)
# This is an LLMChain to write a synopsis given a title of a play and the era it is set in.
llm = OpenAI(temperature=.7)
template = """You are a playwright. Given the title of play and the era it is set in, it is your job to write a synopsis for that title.
Title: {title}
Era: {era}
Playwright: This is a synopsis for the above play:"""
prompt_template = PromptTemplate(input_variables=["title", 'era'], template=template)
synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, output_key="synopsis")
# This is an LLMChain to write a review of a play given a synopsis.
llm = OpenAI(temperature=.7)
template = """You are a play critic from the New York Times. Given the synopsis of play, it is your job to write a review for that play.
Play Synopsis:
{synopsis}
Review from a New York Times play critic of the above play:"""
prompt_template = PromptTemplate(input_variables=["synopsis"], template=template)
review_chain = LLMChain(llm=llm, prompt=prompt_template, output_key="review")
# This is the overall chain where we run these two chains in sequence.
from langchain.chains import SequentialChain
overall_chain = SequentialChain(
chains=[synopsis_chain, review_chain],
input_variables=["era", "title"],
# Here we return multiple variables
output_variables=["synopsis", "review"],
verbose=True)
review = overall_chain({"title":title, "era": era})
print (review)And the output:
> Entering new SequentialChain chain...
> Finished chain.
{'title': 'Tragedy at sunset on the beach', 'era': 'Victorian ', 'synopsis': "\n\nTragedy at Sunset on the Beach is set in the Victorian era and tells the story of a young girl, Amelia, whose search for love and happiness leads her to a fateful encounter with a mysterious stranger on the beach at sunset. While walking along the shore, Amelia meets a man named William, who presents her with a rose as a symbol of his affection. Though she initially resists his advances, Amelia eventually finds herself drawn to William and the two quickly grow close. \n\nHowever, as the sun sets, a secret from William's past is revealed that changes everything. It turns out that William is a wanted criminal and is being pursued by the police. After a tense confrontation, William is arrested and taken away, leaving Amelia heartbroken and alone.\n\nIn the aftermath of their chance encounter, Amelia is forced to grapple with the tragedy of her unrequited love, and the harsh realities of life. Her journey serves as a reminder that love isn't always as simple and straightforward as it may seem.", 'review': '\n\nTragedy at Sunset on the Beach is a captivating tale of love, loss, and the power of the human spirit. The set in the Victorian era allows for a unique perspective on the themes of the story, and the performance by the cast is truly remarkable.\n\nThe story follows Amelia, a young girl whose search for love and happiness leads her to a fateful encounter with a mysterious stranger on the beach at sunset. We watch as Amelia’s relationship with the stranger, William, develops and evolves, only to be cut short when a secret from his past is revealed. This revelation forces Amelia to confront the harsh realities of life and the tragedy of her unrequited love.\n\nThe play is a thought-provoking exploration of love, sacrifice, and the power of human connection. It’s a story that is sure to resonate with audiences who are looking for a story of love, loss, and hope. Highly recommended.'}In Conclusion
In the recent past, I have written a number of times on the coming fragmentation and that a single contained Conversational AI framework will not suffice.
Already we are seeing that LLMs are forcing traditional chatbot framework providers to look out ward to technologies they should make provision for.
LangChain is software that fills a critical voice which currently exists to make LLMs more conversational.
⭐️ Please follow me on LinkedIn for updates on Conversational AI ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
