Automatic Prompt Engineer UI
The Automatic Prompt Engineer (APE) UI demonstrates how prompts can be optimised for text generation.
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
APE takes as input a dataset with a list of inputs and outputs and a prompt template.
Then APE uses a language model to generate a set of possible prompts. APE also has a prompt evaluation function to evaluate the quality of each generated prompt.
APE shows promise and the concept is novel, and can expand, but the following needs to be noted:
- While APE shows promise, human creativity will still plays a crucial role.
- The use-cases from the study are more simplistic and not complicated prompts as one would find at the heart of Autonomous Agents.
- Use-cases excluded in the study were prompt chaining applications or where LLMs accessed external tools.
- Even-though this approach can serve as a stepping stone or a single iteration building to a larger solution, the complexity of prompt evaluation is well documented in a recent released study.
- Below the complete code example to run the APE UI.
#@title Install Dependencies
! git clone https://github.com/keirp/automatic_prompt_engineer.git
! pip install openai numpy tqdm gradio
#@title Import
import os
import sys
import openai
for path in ['/content/automatic_prompt_engineer/automatic_prompt_engineer',
'/content/automatic_prompt_engineer']:
if path not in sys.path:
sys.path.insert(1, path)
from demo import get_demo
openai.api_key = 'xxxxxxxxxxxxxxxxxxxxxxxx'
demo = get_demo()
demo.launch(debug=True, share=True)Considering the image below, when you click on tasks, different datasets are loaded for prompt generation and scoring.
After the dataset is loaded, an estimate of cost can be generated prior to committing the evaluation.
Under configuration there are basic and advanced configuration tabs.
After running the APE functionality, the results are shown below, with the scores.
In conclusion, the APE GUI is a convenient way to play around with datasets to create and tweak prompts.
However, in a follow-up post I will be considering a recent study, which highlighted the challenges and complexities of prompt engineering. And how an AI accelerated latent-space can assist with that.
⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
