Limits - joeylking/agent-runtime GitHub Wiki

Limits

Limits are enforced by the driver, not by asking the model to be careful. Each ends the run with a specific reason and, for the run limits, a limit.exceeded or loop.detected event that names it.

Limit Checked Reason
MaxSteps before each step limit_steps
MaxConsecutiveToolFailures before each step, on the streak of failure observations; a success resets it repeated_tool_failures
LoopThreshold before each step, on identical (tool, arguments, observation) in a row; a changed result is progress loop_detected
MaxModelCalls before each model dispatch, counting retries and ambiguous attempts limit_model_calls
MaxOutputTokensPerCall as the request cap none; it bounds, not ends
MaxTotalTokens before each model dispatch, on totals plus a conservative estimate limit_tokens
MaxEstimatedCost before each model dispatch, on totals plus an estimate, in micro-units limit_cost
MaxActiveTime before each step, on the sum of step durations; approval waits excluded limit_active_time
MaxElapsedTime before each step, as an absolute deadline from creation limit_elapsed_time
ApprovalTTL on approve or resume of a pending approval, from when it was requested approval_expired (run cancelled)
GrantTTL on resume of an approved grant, from when it was decided approval_expired (run cancelled, nothing executed)

DefaultLimits sets 50 steps, three consecutive failures, and a loop threshold of three; the rest are off. MaxSteps, MaxConsecutiveToolFailures, and LoopThreshold must be positive. A zero optional limit is unlimited.

Why projections

Token and cost limits are checked before a call using an overestimate of what the call will use: the totals so far, plus the request's serialized size at one token per three bytes, plus its output cap. A run therefore stops slightly before the limit rather than after crossing it. The estimate is deliberately generous; tune the limit, not the estimate.

An uncapped request is refused under a projected limit. The projection counts the request's output cap, so a request with no cap, from neither its own MaxOutputTokens nor Limits.MaxOutputTokensPerCall, cannot be bounded. Under MaxTotalTokens, or under MaxEstimatedCost for a priced model, it is refused before dispatch as limit_tokens or limit_cost, with a detail saying to set a cap. Set MaxOutputTokensPerCall, which also caps any request asking for more.

Concurrent calls reserve. An agent that calls the model several times at once within one step cannot slip past a limit by having every call check it before any is recorded: each call reserves its projection under a lock before dispatch and releases it once its outcome is added to the run totals.

Totals saturate. Run totals and costs are added with saturating arithmetic, so a provider reporting an absurd count fills them to the maximum, which trips the next check, rather than wrapping negative and passing every check after it. Usage that is not plausible at all, a negative count or more cached input than input, is not charged as reported: the call is charged the conservative estimate and the run fails as model_unavailable. See Models and Replay.

Retries and Retry-After

Retries are a limit too, because each one is a billed attempt. They are off unless ModelConfig.MaxRetries is set: zero means a transient failure ends the run as model_unavailable at once. With it set, a transient error is retried up to that many times, each attempt its own model_calls row counted against MaxModelCalls, with exponential backoff from ModelConfig.Backoff (one second by default).

A provider can ask for a longer wait. All three adapters read Retry-After, in seconds or as an HTTP date, from 408, 409, 425, 429, and 5xx responses into TransientError.RetryAfter; a negative value or one over an hour is treated as absent. The accounting caller waits at least that long, and at least the backoff. It does not wait if the wait cannot fit: a request for more than ModelConfig.MaxRetryAfter (default DefaultMaxRetryAfter, one minute), a wait that would pass the context's deadline, or one that would reach the run's elapsed or active time limit, fails as model_unavailable immediately instead of sleeping. A limit is not crossed by sleeping into it.

Time limits and waits

Active time measures the agent's work and is the right thing to cap for cost. Elapsed time measures the wall clock including days waiting for a human and is the right thing to cap for staleness. A consumer sets either or both.

Both are judged by Config.Now, the driver's clock, before each step. Active time is the sum of step durations by that clock, so a step during which the process was suspended counts in full; time spent waiting for approval is excluded, and a resumed step's clock restarts from the resume. Elapsed time is an absolute deadline from the run's creation and includes every wait.

The two approval clocks are judged when something next touches the approval, by the clock of whoever touches it: the driver's Now for Driver.Approve and Resume, the wall clock for the package-level functions cmd/agentrt uses. GrantTTL is judged only by Resume. A suspension neither extends nor shortens an expiry, and a deterministic Now that does not advance never expires anything. Approvals and Resume compares them.

The run lease is not a limit, but it is the one timer that uses the wall clock and never Now.

Why loop detection keys on the observation

Repeating a call is not a loop if the result changes: re-running validation after a fix is progress. The same call with the same result three times in a row is not, whatever the agent says about it. Steps whose observation was invalid_decision break the sequence, so a model that keeps failing to produce a call is ended by the failure limit, not reported as a loop.

Where a consumer adds its own

Domain budgets, such as a number of validation cycles or a no-progress rule, belong in the policy as abort decisions, where they are visible in the log with a reason.