Limits

The tool calling loop runs until the model decides it is done. Left unchecked, it can run for a very long time: the model calls the same tool over and over, gets stuck in a cycle, or keeps trying variations that all fail.

Three limits#

Iteration cap#

Set a maximum number of tool calls per request. If the model has no final answer after 10 or 15 calls, something has gone wrong. Stop the loop and return an error instead of letting it run on.

Budget cap#

Each tool call is another model pass, with the whole conversation sent again. A loop that runs 20 times costs about 20 times as much as a single call, and more as the conversation grows. Track tokens across the loop and stop when you pass your limit. See Cost for why every pass adds up.

Repetition check#

If the model calls the same tool with the same arguments twice in a row, it is stuck. The second call will get the same result as the first. Break the loop now. Don't wait for the iteration cap.

The loop with all three#

MAX_CALLS = 10
TOKEN_BUDGET = 50_000

def run(messages, tools):
    last_call = None
    for _ in range(MAX_CALLS):
        reply = call_model(messages, tools)
        if reply.tool_call is None:
            return reply.text  # final answer
        call = (reply.tool_call.name, reply.tool_call.args)
        if call == last_call:
            raise StuckError(call)  # same call twice in a row
        if tokens_used(messages) > TOKEN_BUDGET:
            raise BudgetError()
        last_call = call
        messages += [reply.message, run_tool(reply.tool_call)]
    raise TooManyCallsError()

Here call_model, run_tool and tokens_used stand for your own helpers. run_tool is where the checks from Validation happen.

Short line#

A tool calling loop must not run unchecked. Cap the number of tool calls, cap the token budget because every pass costs another model call, and stop at once when the model repeats the same call with the same arguments.

Key takeaways#

• The loop runs until the model stops, so it can run far too long.
• Set an iteration cap, for example 10 to 15 tool calls, and surface an error when it is hit.
• Set a budget cap. Each tool call is another full model pass.
• The same tool with the same arguments twice in a row means the model is stuck. Stop early.