Design, Behaviour & Science
Design relates to decisions made prior to launch. Behaviour surfaces when applications are exposed to the rigours of production. Science underpins the possibilities of LLMs, but not without challenges.
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
A recent paper considered the challenges of creating Generative Applications, also referred to as Gen Apps, which are based on LLMs.
The broader study and findings can be premised on three pillars, Design, Behaviour and Science.
Performance increases through larger compute budgets but at a decreasing rate if the model or dataset size is fixed, reflecting a power law with diminishing returns. ~ Source
This Venn Diagram resonates with me due to the fact that at the centre of the diagram, the commonality shared by design, behaviour and science are tasks which cannot be solved by sheer scale.
There needs to be a level of complexity within a customer facing application where the experience can be tweaked for the user. Hence, at scale, user behaviour needs to be observable, inspectable, measurable and tweakable.
Between design and behaviour are commonly known problems like high inference latency, context length and the presence of LLM hallucination. The more context is supplied, the higher the inference latency and cost; but hallucination is diminished.
LLM inference latencies remain high because of low parallelizability and large memory footprints. ~ Source
The table below, from the same study shows some of the innovations in terms of Prompt Engineering. But the overlap between Science and Behaviour includes observation and inspecting generated text. Hence monitoring prompt performance.
The issue of cost also surfaces quite a bit during the study. The cost of fine-tuning a model, which not only includes the LLM base costs, but data collection, transformation, etc.
It can be argued that Design not only covers application architecture from a technical perspective, but also user experience. And how easy the UX can be adapted to user behaviour as usage grows. Design is also influenced by possible LLM inaccuracy and hallucination, inference latency and cost.
⭐️ Follow me on LinkedIn for updates on Large Language Models ⭐️
I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI & language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces & more.
