LlamaIndex Reasoning & Act (ReAct) Chatbot Engine
How To Build a ReAct-Based Chatbot, A Powerful Approach for AI Agents Over Contained Data.
The LlamaIndex ReAct chatbot project is a standout implementation example, perfectly suited for running in a Notebook and experimentation.
It offers a clear, hands-on way to explore AI Agents and their ability to reason over a compact dataset.
This project is an excellent starting point for understanding how intelligent agents can deliver precise, data-driven responses.
One innovative approach to building intelligent AI Agents is the ReAct (Reasoning and Acting) framework, as demonstrated in the LlamaIndex documentation.
This code notebook looks at the ReAct chatbot approach and why it’s an excellent example of an AI Agent operating over contained data.
What is the ReAct Chatbot Approach?
The ReAct framework combines reasoning and action to create a dynamic chatbot capable of interacting intelligently with users.
Built on top of a query engine tied to a specific dataset, a ReAct chatbot operates in a loop for each user interaction:
- Decision: The agent decides whether to query the underlying knowledge base or rely on its language model capabilities.
- Action: If querying is chosen, the agent formulates an appropriate input for the query engine and processes its output.
- Evaluation: The agent determines whether to repeat the loop (for example, refine the query) or provide a final response to the user.
This loop enables the chatbot to balance querying structured data with generating natural language responses, making it highly flexible and context-aware.
Why ReAct Excels for Contained Data
When working with contained data — such as a company’s internal documents, a product catalog, or a curated knowledge base — the ReAct approach shines. Here’s why:
Flexibility in Querying
Unlike traditional chatbots that may rely solely on pre-trained models or static responses, ReAct dynamically decides whether to fetch information from the knowledge base. This ensures responses are grounded in accurate, domain-specific data when needed.
Reduced Hallucination
By prioritising queries to a trusted dataset, ReAct minimises the risk of the language model generating incorrect or fabricated answers (hallucinations).
Contextual Reasoning
The reasoning step allows the chatbot to evaluate the user’s intent and determine the best course of action, making interactions more relevant and precise.
A Practical Example: ReAct in LlamaIndex
The LlamaIndex example showcases a ReAct chatbot built using the LlamaIndex framework.
In this setup, the chatbot is integrated with a query engine that accesses a predefined dataset.
For instance, imagine a customer support chatbot for an e-commerce platform. When a user asks, What’s the return policy for electronics?, the ReAct agent:
- Recognises the need to query the knowledge base for the specific policy.
- Retrieves the relevant information from the dataset.
- Crafts a natural, user-friendly response based on the query output.
If the user asks a follow-up question like, Can I return an item after 30 days?, the AI Agent might repeat the loop to fetch additional details or clarify based on the existing context. This iterative process ensures high accuracy and relevance.
Benefits of a ReAct Chatbot
The ReAct approach positions the chatbot with agentic capabilities, capable of autonomous decision-making. Key benefits include:
- Scalability: ReAct can handle diverse datasets, from small knowledge bases to large document repositories, making it suitable for various industries.
- Performance Dependency on LLM Quality: While the chatbot’s effectiveness relies on the underlying large language model (LLM), techniques like prompt engineering can optimise its decision-making to favour querying over hallucination.
- User Trust: By grounding responses in verified data, the chatbot builds trust with users, especially in critical applications like healthcare or finance.
Conclusion
The ReAct chatbot approach, as shown in the LlamaIndex documentation, is a compelling solution for building agentic chatbots over contained data.
Its ability to reason, act, and iterate makes it highly effective for delivering accurate, context-aware responses.
Whether you’re creating a customer support bot, a knowledge base assistant, or an internal tool, ReAct offers a robust framework for intelligent, data-driven conversations.
By leveraging tools like LlamaIndex, developers can harness the power of ReAct to build chatbots that are both reactive and reliable, setting a new standard for AI-driven interactions.
Working Notebook
Below is a real simple Notebook which you can run in a Colab environment. All you need is an OpenAI API key and an Anthropic API key.
%pip install llama-index-llms-anthropic
%pip install llama-index-llms-openai!pip install llama-index!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'import os
os.environ["OPENAI_API_KEY"] = "<Your OpeAI API Key>"from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.openai import OpenAI
from llama_index.llms.anthropic import Anthropic
llm = OpenAI()
data = SimpleDirectoryReader(input_dir="./data/paul_graham/").load_data()
index = VectorStoreIndex.from_documents(data)chat_engine = index.as_chat_engine(chat_mode="react", llm=llm, verbose=True)response = chat_engine.chat(
"Use the tool to answer what did Paul Graham do in the summer of 1995?"
)Added user message to memory: Use the tool to answer what
did Paul Graham do in the summer of 1995?
=== Calling Function ===
Calling function: query_engine_tool with
args: {"input":"What did Paul Graham do in the summer of
1995?"}
Got output: In the summer of 1995, Paul Graham started
working on a new version of Arc with Robert. This version
of Arc was compiled into Scheme, and to test it,
Paul Graham wrote Hacker News. Initially meant to be a
news aggregator for startup founders, it was later
renamed Hacker News with a broader topic to engage
intellectual curiosity.
========================print(response)In the summer of 1995, Paul Graham started working on a new
version of Arc with Robert. This version of Arc was compiled
into Scheme, and to test it, Paul Graham wrote Hacker News.
Initially meant to be a news aggregator for startup founders,
it was later renamed Hacker News with a broader topic to
engage intellectual curiosity.llm = Anthropic()chat_engine = index.as_chat_engine(llm=llm, chat_mode="react", verbose=True)import os
os.environ["ANTHROPIC_API_KEY"] = "<Your Antrhopic API Key>"response = chat_engine.chat("what did Paul Graham do in the summer of 1995?")
print(response)Unfortunately, I do not have enough information to state
what Paul Graham did in the summer of 1995. The context
provided to me discusses some of his activities in later
years, but does not give any details about that specific
summer. Without any relevant information, I cannot provide
a factual answer. I apologize that I cannot be more helpful
in answering the exact question asked.response = chat_engine.chat("What did I ask you before?")
print(response)Unfortunately, I do not have access to our previous
conversation history and cannot state the exact question
you asked me before. As an AI assistant without memory of
prior interactions, I can only respond to the current
input provided to me. I apologize that I do not recall
what you asked previously - could you please restate your
original question if you still need an answer? Going forward,
providing some context from any earlier questions will help
me better understand the background.chat_engine.reset()print(response)response = chat_engine.chat("What did I ask you before?")
print(response)Unfortunately, I do not have any record of previous
questions you asked me. As an AI assistant without memory
of prior conversations, I can only respond to the current
question based on the provided context.
Chief Evangelist @ Kore.ai | I’m passionate about exploring the intersection of AI and language. From Language Models, AI Agents to Agentic Applications, Development Frameworks & Data-Centric Productivity Tools, I share insights and ideas on how these technologies are shaping the future.
