The History & Future of Prompt Engineering

A recently published research paper on arXiv was entirely authored by ChatGPT 3.5 Turbo and ChatGPT.

3 min readSep 3, 2024

--

Introduction

A newly published paper delves into the ๐—ฒ๐˜ƒ๐—ผ๐—น๐˜‚๐˜๐—ถ๐—ผ๐—ป ๐—ผ๐—ณ ๐—ฝ๐—ฟ๐—ผ๐—บ๐—ฝ๐˜ ๐—ฒ๐—ป๐—ด๐—ถ๐—ป๐—ฒ๐—ฒ๐—ฟ๐—ถ๐—ป๐—ด, revealing how far weโ€™ve come and whatโ€™s on the horizon.

Iโ€™ve transformed this studyโ€™s data into a visual snapshot that highlights both past ๐—ฎ๐—ฑ๐˜ƒ๐—ฎ๐—ป๐—ฐ๐—ฒ๐—บ๐—ฒ๐—ป๐˜๐˜€ and ๐—ณ๐˜‚๐˜๐˜‚๐—ฟ๐—ฒ trends.

Since 2022, weโ€™ve seen a shift from traditional language generation to innovative techniques like ๐—บ๐˜‚๐—น๐˜๐—ถ๐—บ๐—ผ๐—ฑ๐—ฎ๐—น ๐—ฝ๐—ฟ๐—ผ๐—บ๐—ฝ๐˜๐—ถ๐—ป๐—ด, ๐—บ๐˜‚๐—น๐˜๐—ถ-๐˜๐˜‚๐—ฟ๐—ป ๐—ฐ๐—ผ๐—ป๐˜ƒ๐—ฒ๐—ฟ๐˜€๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€, and ๐—ฑ๐—ผ๐—บ๐—ฎ๐—ถ๐—ป-๐˜€๐—ฝ๐—ฒ๐—ฐ๐—ถ๐—ณ๐—ถ๐—ฐ ๐—ธ๐—ป๐—ผ๐˜„๐—น๐—ฒ๐—ฑ๐—ด๐—ฒ ๐—ถ๐—ป๐˜๐—ฒ๐—ด๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป.

Key future developments include ๐—ฟ๐—ฒ๐—ถ๐—ป๐—ณ๐—ผ๐—ฟ๐—ฐ๐—ฒ๐—บ๐—ฒ๐—ป๐˜ ๐—น๐—ฒ๐—ฎ๐—ฟ๐—ป๐—ถ๐—ป๐—ด ๐˜„๐—ถ๐˜๐—ต ๐—ต๐˜‚๐—บ๐—ฎ๐—ป ๐—ณ๐—ฒ๐—ฒ๐—ฑ๐—ฏ๐—ฎ๐—ฐ๐—ธ, better ๐—ฐ๐—ผ๐—ป๐˜๐—ฒ๐˜…๐˜๐˜‚๐—ฎ๐—น ๐—ถ๐—ป๐˜๐—ฒ๐—ป๐˜ prediction, and more personalised user modelling.

Authors extensively used ChatGPT for content generation. ~ Source

The Prompts Used

Prompt #1: You are a scholar in machine learning and lan- guage models. I am writing a paper on the history of prompt engineering and generation. Can you give me a timeline for prompt engineering evolution? (We used this timeline to create prompts for each section later)

Prompt #2: Write the introduction of this paper. Emphasize that this paper focuses on how language prompts and queries have been used so far.

Prompt #3: Now generate history of prompting or querying in early language models an information retrieval

Prompt #4: Now write the history between 2010 and 2015 before attention mechanism was invented

Prompt #5: now write a section on how attention mechanism changed the future of prompt engineering in 2015

Prompt #6: now discuss how the advent of reinforcement learning techniques in 2017 changed the prompt engineering

Prompt #7: (a) Now write the section on developments in prompt engineering in 2019. (b) Now can you rewrite the section on developments in prompt engineering in 2019? Please organize your thoughts in paragraphs instead of bullet points.

Prompt #8: (a) now write the section for 2020 and 2021 in prompt engineering (b) now rewrite the section for 2020 and 2021 in prompt engineering? Please organize your thoughts in paragraphs instead of bullet points

Prompt #9: (a) Can you now write a section on 2022 and 2023 on advanced prompt techniques? (b) can you write the section on 2022 and 2023 on advanced prompt techniques in paragraphs instead of bullet points?

--

--

Iโ€™m passionate about exploring the intersection of AI & language. www.cobusgreyling.com