Can Minor Document Typos Comprehensively Disrupt RAG Retriever & Reader Components?

Retrieval-Augmented Generation (RAG) is effective in leveraging LLM in-context learning (ICL) capabilities. But are we missing how different parts of RAG work together & ignore real-world challenges like small mistakes in data?

Introduction

This recent study focusses on two areas:

1. How RAG handles noisy documents, &
2. A complete review of RAG’s strengths.

The study introduces a new attack, the Genetic Attack on RAG (GARAG), to test these areas. GARAG finds weaknesses in each part of RAG and tests the whole system with noisy documents.

The study proves RAG’s strength by using GARAG on common QA datasets with various retrievers and LLMs. The results show GARAG consistently succeeds in attacking RAG, exposing risks from small errors in real-world data.

The image shows the Impact of the noisy RAG documents.

Key Findings From The Study

Three key findings from the study:

• They point out that RAG systems are vulnerable to minor but frequent textual errors within the documents.
• An attack method called GARAG is proposed, based on an algorithm searching for adversarial documents.
• RAG systems are susceptible to noisy documents in real-world databases.

The reader’s ability to accurately ground information significantly depends on the retriever’s capability of sourcing query-relevant documents.

Genetic Attack on RAG (GARAG)

GARAG assesses the holistic robustness of a RAG system against minor textual errors, offering insights into the system’s resilience through iterative adversarial refinement.

In Conclusion

This new study contains three main contributions…

1. Highlighting a vulnerability in RAG systems pertaining to frequent minor textual errors within documents. This evaluation focuses on the retriever and reader components’ functionality.
2. Introducing GARAG, a straightforward & potent attack strategy leveraging a genetic algorithm to craft adversarial documents capable of exploiting weaknesses in both components of RAG simultaneously.
3. Through experimentation, demonstrating the detrimental impact of noisy documents on the RAG system within real-world databases.

The results show that typos seriously harm the RAG system, making it work much worse. Even though the retriever helps protect the reader, it can still be affected by small disruptions.

⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️

I’m currently the Chief Evangelist @ Kore AI. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.

LinkedIn

Get an email whenever Cobus Greyling publishes.

Cobus Greyling | Substack

Cobus Greyling

Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild…