Agents are powerful for open-ended tasks. That power has a price, and it is paid on every request.
What an agent costs you#
| Workflow | Agent | |
|---|---|---|
| Open-ended tasks | Weak: steps must be known upfront | Strong: steps are chosen at runtime |
| Cost | Fixed number of calls | Multiplies with each iteration |
| Latency | Fixed number of waits | Multiplies with each iteration |
| Predictability | Same steps every time | Steps vary per request |
| Debugging | Find the step that failed | Trace back through several model decisions |
Cost and latency#
Each iteration is another model call, with the whole conversation sent again, and another wait. An agent that takes 12 steps costs and waits roughly 12 times as much as one call. See Cost.
Debugging#
When a workflow fails, you find the step that broke. When an agent fails, the model made several decisions that led there: which tool, which arguments, when to continue. Tracing back through those decisions takes more work. Logging every step of the loop makes it possible.
When an agent is the wrong choice#
- You can name all the steps upfront.
- The task is the same shape every time.
- Cost or latency limits are tight.
- You need the same behavior on every run, for example for audits or strict tests.
In each case a workflow does the job for less.
Start with a workflow#
Start with a workflow. If it cannot handle the task, because the steps genuinely cannot be decided beforehand, then introduce an agent. Most things that look like they need an agent can be solved with a well-designed workflow.
Short line#
Agents handle open-ended tasks, but cost, latency and debugging effort all grow with every iteration. Start with a workflow, and bring in an agent only when the steps truly cannot be decided beforehand.
Key takeaways#
- Agents are flexible for open-ended tasks.
- Cost and latency multiply with each iteration.
- Debugging is harder because several model decisions led to the failure.
- An agent is the wrong choice when the steps are known, the task has a fixed shape, or cost and latency are tight.
- Start with a workflow. Add an agent only when the steps cannot be predetermined.