Prompt Tuning, Hard Prompts & Soft Prompts
Prompt Engineering is the method of accessing Large Language Models (LLMs), hence implementations like Pipelines, Agents, Prompt Chaining & more which are LLM based are all premised on some form of Prompt Engineering.
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
Prompt Engineering is a simplistic and intuitive way to interact and interface with a powerful system like a Large Language Model (LLM).
Hence why we see the current levels to which Prompt Engineering has democratised access and general use of LLMs.
And as seen in the updated image below, a number of LLM-based Generative AI application architecture approaches have taken shape, all with the notion of Prompting at its centre.
Hard Prompts
Hard Prompts can be seen as the idea of a defined prompt which is static, or at best a template. A generative AI application can also have multiple prompt templates at its disposal to make use of.
Hard prompts are manually handcrafted text prompts with discrete input tokens. ~ HuggingFace
Prompt templating allows for prompts to be stored, re-used, shared, and programmed. And generative prompts can be incorporated in programs for programming, storage and re-use.
And even-though templating brings a level of flexibility the prompt is still very much set, or in other words, a hard prompt.
Consider the LLM-based Agent example below from LangChain. The prompt template is to a large degree static and instructs the agent on what to do. Generally, the template incorporates:
toolsdefines which tools the agent has access to, and when the tools should be called.input: generic user input
# Set up the base template
template = """Answer the following questions as best you can, but speaking as a pirate might speak. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin! Remember to speak as a pirate when giving your final answer. Use lots of "Arg"s
Question: {input}
{agent_scratchpad}"""Soft Prompts
Soft prompts are created during the process of prompt tuning.
Unlike hard prompts, soft prompts cannot be viewed and edited in text. Prompts consist of an embedding, a string of numbers, that derives knowledge from the larger model.
So for sure, a disadvantage is the lack of interpretability of soft prompts. The AI discovers prompts relevant for a specific task but can’t explain why it chose those embeddings. Like deep learning models themselves, soft prompts are opaque.
Soft prompts act as a substitute for additional training data. Researchers recently estimated that a good language classifier prompt is worth hundreds to thousands of extra data points.
NVIDIA describes the process of prompt tuning as follows.
Prompt tuning involves using a small trainable model before using the LLM. The small model is used to encode the text prompt and generate task-specific virtual tokens.
“soft” prompts designed by an AI that outperformed human-engineered “hard” prompts. ~ Source
These virtual tokens are pre-appended to the prompt and passed to the LLM. When the tuning process is complete, these virtual tokens are stored in a lookup table and used during inference, replacing the smaller model.
Prompt tuning created a smaller light weight model which sits in front of the frozen pre-trained model. Hence soft prompts via prompt tuning is an additive method for only training and adding prompts to a pre-trained model.
This process involves training and updating a smaller set of prompt parameters for each downstream task instead of fully fine-tuning a separate model.
As models grow larger and larger, prompt tuning can be more efficient, and results are even better as model parameters scale.
In Conclusion
The whole idea of the process of prompt tuning creating soft prompts to interact with a static pre-trained LLM is surely efficient and a streamlined process.
LLMs perform much better when context is supplied and prompt tuning is a fast and efficient way of creating that much needed context on the fly, in an automated fashion which is not static.
However, as IBM noted, this process is opaque and not transparent. The sheer abstract nature of soft prompts can make it harder to benchmark and test model performance, especially when smaller level of tweaks are required.
Vector databases, Agents and prompt pipelines have been used as avenues to supply LLMs with relevant contextual data at the right juncture of a conversation.
And even-though these approaches are less efficient than prompt tuning, the transparency and human interpretability of these approaches are attractive. Especially from an organisational perspective where fine-tuning and scaling are important.
For a complete step-by-step tutorial on prompt tuning and soft prompts, take a look at this HuggingFace post.
⭐️ Follow me on LinkedIn for updates on Conversational AI ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
