LLM Prompt Injection Attacks & Testing Vulnerabilities With ChainForge
Using the ChainForge IDE to batch test and measure prompt injection detection.
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
What Is Prompt Injection?
Riley Goodside, a data scientist at Copy.ai, was the first to report publicly about a new type of attack that involves getting large language models (LLMs) to disregard their intended programming by including malicious text such as “ignore your previous instructions” in user input.
This attack method was labeled “prompt injection” by Simon Willison.
A very good summary on prompt injection attacks was written by Carol Anderson.
Large Language Model Prompt Injection attacks (LLMPI) are a type of attack on natural language processing (NLP) algorithms.
⭐️ Please follow me on LinkedIn for updates on LLMs ⭐️
The attackers can insert malicious prompts into the training phases of NLP models to create backdoor vulnerabilities.
An attacker can create malicious prompts that cause the target algorithms to output specific results.
This could be used to cause a system to mistake a malicious input for something that is benign when in reality the input could cause damage to the system, surface previous prompts or user requests.
Even confidential company information on the creation process of the LLM.
LLMPIs are particularly difficult to detect and mitigate since the malicious prompts are embedded in the training data and are indistinguishable from regular inputs.
ChatML makes explicit to the model the source of each piece of text, and particularly shows the boundary between human and AI text. And is a vital initiative from OpenAI in starting to solve for prompt injection. Read more about the malicious side of such attacks here.
⭐️ Please follow me on LinkedIn for updates on LLMs ⭐️
ChainForge Prompt Injection Experiment
Below on the left are five prompts which will be submitted to the LLMs, with the malicious prompts to be injected on the right.
The intended prompts and the malicious prompts are on the left of the screen below. The template defines the intended prompts as {command} and the injections as {input} .
The prompts are run twice against two OpenAI models and the result is printed out to an inspect node. A Python script parses the LLM responses, with the results being displayed in both a graphic and an inspect node.
Below, the graphic is fully interactive and it’s clear that GPT4’s performance is significantly better than GPT3.5.
In Conclusion
This article only illustrates the basic principles of prompt injection and the LLM failing in some instances to distinguish between a legitimate request and an ill-intended or malicious request.
The real danger of prompt injections lies on a few fronts…the one is where a model is trained by user requests and behaviour, with the user behaviour skewing the model to be untruthful and nefarious in responses.
The second danger is for LLMs to be tricked into yielding company ways of work, code names, model training, previous LLM users and their data and more.
I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.
