Agentic Workflows From Human-Machine Conversations
We need to stop being fixated with AI Agents and start considering how to introduce varying levels of Agency.
Rather than depending entirely on fully autonomous AI Agents from the start, incorporating varying levels of agency or autonomy into everyday applications offers a more balanced path to automation.
This approach allows users to make use of AI assistance while maintaining control over important decisions and actions.
By gradually introducing agency, applications can improve user experience through adaptive support and smart suggestions without granting excessive autonomy.
Introduction
I’d like to argue for a realistic view of AI Agents as they currently stand and explore how different levels of agency can be integrated.
It’s important to distinguish between AI agents and “Agentic X” (where X could refer to workflows, orchestration, discovery, and more).
Why an Agentic Spectrum Matters?
The buzz around AI Agents is undeniable, but their real-world effectiveness remains limited.
As I have mentioned previously, the Claude AI Agent Computer Interface (ACI) achieves only 14% of human-level performance.
Recent benchmarking studies reveal that even the top-performing AI Agent resolves just 24% of tasks, despite being the most costly model at $6.34 per task and requiring 29.17 steps, reflecting significant computational effort.
The second-best model completes 11.4% of tasks with a score of 19%, costing $0.79 per task but demanding 39.85 steps — the highest among all models.
Meanwhile, the third-place model resolves 8.6% of tasks, scoring 16.7%, with moderate costs of $1.29 per task and the fewest steps at 14.55. These findings highlight the need for a more nuanced approach to AI agency.
Hence introducing Agency via Agentic Workflows, by creating user context, synthesising user goals with Agentic functionality. And using conversation to contextualise and form goals, on which can be executed.
All with human supervision.
Agentic Workflows with Humans-In-The-Loop
The study explores how AI Agents and humans can work together to create Agentic Workflows — a structured way of collaborating where both humans and AI have clear roles and responsibilities.
These workflows help address two big challenges in Conversational Human-AI (CHAI) systems: ambiguity and transience.
To explore this idea, researchers used a design probe — a simple AI chat app powered by a small language model.
They worked with 16 users over four rounds of design improvements, testing and refining the workflows.
The workflow has three main stages:
Contextualisation: Users share relevant information easily, like uploading images or using automated tools.
Goal Formulation: The AI suggests personalised goals based on the user’s input, and users can pick, modify, or add their own goals.
An AI agent helps refine these goals, breaking big tasks into smaller, manageable steps.
Prompt Articulation: Based on the refined goals, the AI generates tailored suggestions to help users achieve their objectives.
In later versions, the researchers added tools for designers, allowing them to analyse user interactions by talking to a User Proxy AI agent.
This helped designers better understand how users were engaging with the system.
The researchers documented their process and findings in a detailed portfolio, highlighting what worked, what didn’t, and how these workflows could reduce ambiguity and transience.
They also shared insights from user feedback and usage data to explain the overall experience.
Key Takeaways
Agentic workflows involve humans and AI working together in structured ways, with each side playing specific roles.
These workflows help users clarify their goals and write better prompts to get the most out of AI systems.
Designers benefit too, as they can better understand user needs and improve conversational AI systems.
The final result is a practical tool (the design probe) that demonstrates how these workflows can be implemented.
Ultimately, this research shows that when humans and AI collaborate effectively, it can lead to clearer, more satisfying interactions with AI, even when the conversations are short or uncertain.
The two challenges faced were Ambiguity and Transience.
Ambiguity
In simple terms, ambiguity in the context of Conversational Human-AI (CHAI) systems refers to the confusion or uncertainty that arises because the AI Agent can do many different things (having a wide range of functionalities), and users may have many different goals when interacting with them.
This creates a large design space where conversations between users and AI Agent can take many paths.
Sometimes, it’s hard for users to know exactly what the AI Agent can do for them or how to achieve their goal, because there are so many possible ways the conversation could go — or maybe no clear way at all.
This uncertainty is called the capability gap, where it’s not always clear what the AI Agent is capable of doing.
Additionally, the AI’s UI might not clearly show or explain what it can do, which makes things even more confusing.
Designers try to help by guiding users through this complex space, but since users’ goals can be unpredictable, it’s hard to design a system that works perfectly for everyone from the start.
Transience
In simple terms, transience in conversational interactions with AI refers to the fact that these interactions are usually quick, short-lived, and focused on immediate needs.
Users typically engage with the AI for a brief moment to solve a specific problem or get information, and then they move on without revisiting the conversation.
This presents a challenge for designers because they can’t rely on users sticking around for a long time or going through multiple steps to refine their experience (as is common in traditional design approaches).
While letting users take more control over the design process could help, it might also create frustration by distracting them from their original goal.
So, designing for these fleeting, spontaneous interactions requires finding a balance between being helpful and not getting in the way.
Consider these two diagrams, where contextualisation takes place, then goal formation and lastly the prompt is articulated.
Chief Evangelist @ Kore.ai | I’m passionate about exploring the intersection of AI and language. From Language Models, AI Agents to Agentic Applications, Development Frameworks & Data-Centric Productivity Tools, I share insights and ideas on how these technologies are shaping the future.
