Rendering Model Context - joeylking/agent-runtime GitHub Wiki

Rendering Model Context

A model-driven agent does two things either side of a model call: turn the run's recorded steps into messages, and turn the response back into a decision. Both consumers wrote that code, and the interesting parts — the recent-results window, the content cap, what to say to a model that asked for nothing — were the same in both. The render package is that half, in the core module with no dependency beyond the standard library. What is left to the consumer is the opening message and the hooks.

func (a *agent) Decide(ctx context.Context, in agentrt.StepInput) (agentrt.Decision, error) {
    msgs := render.Messages(in, render.Options{Opening: a.opening})
    resp, err := in.Model.Generate(ctx, agentrt.ModelRequest{
        System: system, Messages: msgs, Tools: in.Tools, MaxOutputTokens: 4096,
    })
    if err != nil {
        return agentrt.Decision{}, err
    }
    return render.Decide(resp), nil
}

What Messages produces

An opening user message, then two messages per recorded step:

user       the opening: the goal and the domain facts
assistant  the decision's reason as text, then a tool_use block
user       a tool_result block: the observation, with an error flag when it failed
assistant  ...

The tool_use id is synthesized, because the runtime records a decision, not a provider message: by default step_<index>, which is stable for a given step whichever process renders it. That stability is what makes the audit log and the model's view the same data, and it is why a resumed run in a new process renders exactly what a continuous one would.

Steps with no recorded decision are skipped: the step being decided, and any step interrupted before it decided. A step whose decision was not a tool call still appears: there is no tool_use block to render, so the assistant turn is the reason text alone (or (no tool call) when there is none) and the user turn is a nudge.

Arguments that were not JSON. Since v0.3 the runtime records such arguments out of band, in Decision.InvalidArgs (base64 with InvalidBase64 when they were not UTF-8), with Args left empty, so the record of a call that could not be parsed never looks like one that could. render.Args(decision) is what the tool_use block carries: the arguments, {} when there were none, and for an out-of-band value the object {"invalid_json": ...} (or "invalid_json_base64") that earlier releases stored in Args itself. A rendered conversation, and every replay key taken over one, is therefore exactly what it was. A custom Assistant hook that builds its own tool_use block should use render.Args for the same reason.

The recent window and the content cap

Two numbers keep a long run's context from growing without bound:

Option Default Effect
RecentResults 6 (DefaultRecentResults) results older than the last six rendered steps are replaced by [summary] and the observation's summary
MaxContentBytes 24000 (DefaultMaxContentBytes) a result larger than this is replaced by its summary even inside the window

The window counts rendered steps. A step skipped for having no decision does not take a place in it, so a run with an interrupted-while-deciding step still shows the last six results in full. (Before v0.3 the window counted raw steps, and such a step pushed a result out early.)

Both rely on tools writing a useful Summary, which is the practical reason Writing Tools asks for one. The decisions stay in full: what the agent asked for is small, since Decide caps the reason, and is the part a model needs to see to know what it already tried. Only results shrink.

An interrupted observation is rendered as neither content nor summary but as a re-request, render.InterruptedText: the process died while the tool ran and its outcome is unknown, so the model is told to ask again and the tool answers from its own record. The runtime never re-executes a side effect on the agent's behalf, and since v0.3 an interrupted side effect that is not read-only does not reach the agent at all without a reconciliation or an operator's approval; what the agent sees this way is a read-only call, or a step the consumer's reconciliation let continue. See Interruption and Reconciliation.

The hooks

Only Opening is required — Messages panics without it, because a renderer with no domain text is a bug rather than a default. Every other hook may be nil.

Option Purpose Falling back
Opening(StepInput) string the first user message: the goal and the domain facts, and the only place they belong required
ToolUseID(Step) string name a step's tool_use block step_<index>
Assistant(Step, id) []ContentBlock render a step's assistant turn returning nil synthesizes it
Observation(Step, id, full) ContentBlock render the user turn answering a step returning a block with no type uses the default

Assistant and Observation fall back per step rather than per agent, so a hook overrides only the steps it knows about and the default handles the rest. That is what lets a consumer with provider-specific turns for some steps keep the generic rendering for the others.

Both existing renderers are reproduced byte for byte by these options, which is how the extraction was proved faithful. Casework's is the default plus its own ids and raw assistant turns, because its Anthropic runs carry signed thinking blocks that must be replayed as the provider wrote them:

render.Options{
    Opening:       func(agentrt.StepInput) string { return opening },
    RecentResults: a.RecentResults,
    ToolUseID: func(st agentrt.Step) string {
        if t, ok := turns[st.Index]; ok && t.Model == a.ModelName {
            if id := firstToolUseID(t.Raw); id != "" {
                return id
            }
        }
        return fmt.Sprintf("toolu_step_%d", st.Index)
    },
    Assistant: func(st agentrt.Step, _ string) []agentrt.ContentBlock {
        if t, ok := turns[st.Index]; ok && t.Model == a.ModelName {
            return []agentrt.ContentBlock{{Type: RawAssistantBlock, Text: string(t.Raw)}}
        }
        return nil // synthesize
    },
}

Repo Steward keeps the default ids and window and overrides both turns, because its assistant turn carries no reason text and its results are rendered as a labelled head plus content:

calls := func(st agentrt.Step) bool {
    return st.Decision.Kind == agentrt.DecideToolCall && st.Decision.Tool != ""
}
render.Options{
    Opening:       a.opening,
    RecentResults: a.RecentResults,
    Assistant: func(st agentrt.Step, id string) []agentrt.ContentBlock {
        if !calls(st) {
            return nil // the default text turn
        }
        return []agentrt.ContentBlock{{Type: "tool_use", ToolUseID: id, Name: st.Decision.Tool, Input: orEmpty(st.Decision.Args)}}
    },
    Observation: func(st agentrt.Step, id string, full bool) agentrt.ContentBlock {
        if !calls(st) {
            return agentrt.ContentBlock{} // the default nudge
        }
        return agentrt.ContentBlock{Type: "tool_result", ToolUseID: id, Name: st.Decision.Tool,
            Content: renderObservation(st, full), IsError: st.Observation != nil && st.Observation.Failure()}
    },
}

The guards are not optional. Because Decide records no_tool_call and truncated decisions, every consumer's hooks are called for steps that called nothing, and returning the zero values there is what lets the renderer deliver the nudge. A hook that formats results its own way can still reuse the runtime's handling for the steps it does not care about by calling render.DefaultObservation(st, id, full, maxBytes), which also accepts a step without a decision or without an observation and answers it with the nudge. The interruption re-request sentence is exported as render.InterruptedText so a custom format keeps the one sentence the interruption contract depends on.

A consumer targeting the Anthropic adapter gets the raw block's type from Model.RawAssistantBlock(); see Provider Adapters.

Decide and the two pseudo kinds

render.Decide maps a response onto a decision. The stop reasons it reads are exported as constants: StopToolUse, StopEndTurn, StopMaxTokens, StopRefusal, and StopContextWindowExceeded (model_context_window_exceeded).

Stop reason Decision
refusal fail: a model that declined will decline again, so the run ends rather than spending steps on it
model_context_window_exceeded fail: the conversation no longer fits the model's context window, and it only grows. Before v0.3 it was treated as a truncation
max_tokens truncated (agentrt.KindTruncated), even when the reply carries a complete tool use
anything, with no tool use no_tool_call (agentrt.KindNoToolCall), with the model's text as the reason
anything, with a tool use tool_call from the first tool use, arguments defaulting to {}, the model's text as the reason

The reason is capped on every branch at ReasonBytes (500), after it is composed, so a long narration cannot dominate the next render through the reason field.

Tool uses that were not executed are named. Only the first tool use executes. When a reply carries more, the reason says how many were not executed and names them, [2 further tool call(s) not executed: write_file, delete_file], each name capped at NameBytes (64) and the note at half of ReasonBytes, with names that do not fit elided. The note is composed first and the model's text gets what is left of the cap, so the model is always told what did not run. The adapters switch parallel calls off where the provider allows it, but a server may ignore that, and Ollama has no such switch.

These two are the cases the release notes name as changing what render produces for a recorded step, and so a replay key: a reply with more than one tool use, and a reply with no tool use whose text is longer than 500 bytes. The window change above can too, for a run holding a step with no decision. See Upgrading to v0.3.

truncated and no_tool_call are decision kinds the runtime defines and deliberately refuses to validate. The driver records the decision verbatim, validateDecision rejects the unknown kind, and the step ends with an invalid_decision observation naming what was wrong. The next Messages reads the recorded kind and renders a nudge in place of a tool result:

  • no_tool_call: "Your reply contained no executable tool call. Respond with exactly one tool call."
  • truncated: "Your reply hit the output token cap before the tool call was complete, so nothing ran. Respond with exactly one short tool call."
  • anything else the driver found invalid, such as arguments that failed the schema: a generic line carrying the observation's summary.

Note that Decide never produces a complete decision. A model-driven run finishes on a terminal tool, which means the result the run completes with came from a tool that ran, not from the model's say-so.

Why a nudge and not an in-band retry

The obvious alternative is for the agent to call the model again inside Decide: ask a second time, and give up after a few attempts. Recording the failure as a step instead buys three things:

It is auditable. The step is in the log with the model's reply, the pseudo kind, and the invalid_decision observation. An operator reading a run can see that the model failed to produce a call, how often, and what it said instead. A retry inside Decide leaves the model_calls rows but no step, so the run's history shows a gap the log cannot explain.

It counts against the failure limit. An invalid decision is a failure observation, so MaxConsecutiveToolFailures ends a run whose model cannot produce a tool call, with reason repeated_tool_failures, and MaxSteps bounds it besides. An agent looping internally is bounded only by whatever counter it wrote itself, and a bug in that counter is a run that never ends. Limits belong to the driver, not to the agent: see Limits.

No partial tool call ever executes. The adapters drop the tool uses from a truncated or refused reply, and Decide turns what is left into a kind the driver will not act on. There is no path from a half-written call to the policy, let alone to a tool. A retry would have to be careful not to act on the first, broken response; a pseudo kind cannot act on it at all.

The cost is one step and one model call per failure, which is the point: a model that cannot produce a tool call costs a measured, bounded amount before the run stops.

One detail worth knowing: loop detection ignores steps whose observation was invalid_decision, so a repeated failure to produce a call is ended by the failure limit rather than reported as a loop. Loop detection is about a call that keeps producing the same result, and there was no call here.

Truncate

Truncate(s, n) shortens a string to at most n bytes, the "…" that marks the cut included, without splitting a character. When n is too small to hold the mark, the cut is unmarked; a negative n is zero. A string already within n is returned unchanged. (Before v0.3 the mark was appended after cutting to n, so the result could exceed n by the mark's three bytes.) Decide applies it with ReasonBytes. It is exported because a consumer capping its own text wants the same rune-safe behaviour and the same bound.

Next

Provider Adapters is the other half. Writing an Agent puts both together. Getting Started runs examples/live, which is this package plus four tools and a policy.

⚠️ **GitHub.com Fallback** ⚠️