Operator Command - joeylking/agent-runtime GitHub Wiki

The Operator Command

cmd/agentrt is the operator surface over a runs database: list runs, show one, print its audit log, approve, reject, or cancel, and, since v0.5.0, check the audit log's hash chain. It reads and it decides; it never executes a step, never creates a database, and never migrates one. Both consumers had written the same command, so it is in the core module and adds no dependency.

go run ./cmd/agentrt -db runs.db runs
go install github.com/joeylking/agent-runtime/cmd/[email protected]

Build it from the same release as the consumer that owns the database. It refuses a database on any other schema version and says which side is newer. In particular a v0.4.0 binary refuses a database v0.5.0 has migrated, and stopping every older binary before the first open is part of Upgrading to v0.5.

Nothing in it is privileged. It calls the same package-level ApproveShown, RejectShown, and Cancel that take a store and an observer rather than a driver, which is precisely why an operator can decide an approval from a machine that has no copy of the consumer's agent, tools, or policy. That also means whoever can write the database can approve: -by is a label, not a login.

Invocation

agentrt [-db path] [-json] <command> [arguments]
Global flag Effect
-db path the SQLite database; falls back to AGENTRT_DB
-json emit JSON instead of aligned text

The global flags go before the command. approve, reject, and cancel take their own flags before or after the run id; show and events take theirs after it; runs takes them directly.

With neither -db nor AGENTRT_DB the command is a usage error, not a guess at a default path: writing to the wrong runs database is worse than being told to name one. help and -h print the usage text to stdout and exit 0. Every argument is validated before the database is touched, so a typo in a flag or a missing run id never opens the file.

Opening the database

agentrt opens with agentrt.OpenExisting, never OpenStore:

  • Existing databases only. A path that does not exist is an error at exit 1, not a new empty file.
  • Read-only for reading. runs, show, and events open the file read-only: they never write to it and never change its mode. approve, reject, and cancel open it for writing.
  • The file named, not a link to it. A symbolic link is refused rather than followed, and so is anything else that is not a regular file.
  • No migration. A database whose schema version is not this build's is refused.

Each refusal the runtime defines is turned into one plain sentence at exit 1, matched with errors.Is or errors.As, never by text:

Refusal What agentrt prints
a database or WAL file group or others can write (ErrInsecureMode) error: agentrt: runs.db is -rw-rw-r--, writable by group or others; the runtime keeps its database readable and writable by its owner only: chmod 600 runs.db and chown it to the user that runs the runtime
a symbolic link (ErrSymlink) error: agentrt: database path is a symbolic link: link.db; name the database file itself
a trigger or a view in the file (ErrUnsafeSchema) the error, then: a trigger or a view could run SQL you did not write inside agentrt's own statements, so it is refused rather than opened
another schema version (ErrSchemaVersion) the error, then which side is newer (the database or this build), then that agentrt does not migrate a database and its owner is responsible for migrations

Exit codes

Exit Means
0 the command did what was asked
1 an error: no such database or run, a refused database, an approval that is not pending, an approval that changed after it was shown, a confirmation declined at the prompt, a terminal run that cannot be cancelled, an event chain that is broken, a verify -to the chain does not reach, or a verify -head that is not reached or not matched
2 a usage problem: no database named, an unknown command, a missing run id, a trailing argument, a bad flag, approve or reject with no way to confirm, an empty -by with $USER unset. The usage text is printed to stderr with it

The distinction is for scripts. A 2 means the invocation was wrong and retrying it will fail the same way; a 1 means the invocation was right and the state, or the operator, said no.

Text is sanitised

Goals, tool names, arguments, presentations, summaries, and reasons come from a model, a policy, or a remote server. Every such field printed in text mode passes through trace.Sanitize, which shows control characters, bidirectional overrides and isolates, zero-width and other invisible formatting characters, the line and paragraph separators, and the Unicode tag block as visible escapes, and cuts more than four combining marks on one base character with a count of how many were dropped. Ordinary non-Latin scripts and single-codepoint emoji print unchanged; an emoji built with zero-width joiners shows the joiner as an escape. A tool result cannot clear the screen, move the cursor, forge an approval block, or display as other text than it is.

-json output is not sanitised, because a program parsing JSON is not a terminal, but an approval's capability, presentation, and arguments are bounded the same way as in text (see approve below).

Timestamps carry a date and a zone: 2006-01-02 15:04:05 -0700 in local time.

Commands

runs

agentrt -db runs.db runs [-limit n] [-offset n]

Runs newest first, by insertion order: id, status, terminal reason, step count, creation time, the id of the waiting approval if there is exactly one, and the goal clipped to 60 characters. no runs when the database is empty.

RUN                               STATUS                REASON  STEPS  CREATED                    APPROVAL                          GOAL
164ba4c9245839941e291b3d6e71bac5  WAITING_FOR_APPROVAL  -       1      2026-10-01 21:59:42 -0500  7242f85737a3600a21d31fe015538671  add five to the counter

It reads only the page it prints. -limit defaults to 100 and -offset to 0; -limit 0 reads everything, a bounded page at a time. When there are more, the last line says how many and the -offset to see them. -json emits {"runs": [...], "total": n, "limit": n, "offset": n} with "truncated": true when there are more; each run is a view.RunSummary.

show <run>

agentrt -db runs.db show <run> [-limit n] [-offset n]

The run, then every pending approval, then its steps. The header carries the id, the goal, the status with its reason and reason detail, the step count, and the creation and finish times; the goal and the detail are capped at 4000 bytes each. When the run made model calls it also carries a model line: the call count, input and output tokens, and the estimated cost to four decimal places.

Every pending approval is printed before the steps, because it is the only thing on the page that needs a decision:

APPROVAL WAITING  7242f85737a3600a21d31fe015538671
kind    counter_change
tool    add
expires 2026-10-01 22:59:42 -0500
arguments (the tool call the model made; one field per line, byte count is the field's own size)
  n (1 bytes)
    5
presentation (supplied by the consumer's policy; it may contain model-chosen text)
  question (35 bytes)
    "May the agent change the counter?"
decide with: agentrt -db "runs.db" approve 164ba4c9245839941e291b3d6e71bac5 -approval 7242f85737a3600a21d31fe015538671
----- end of approval 7242f85737a3600a21d31fe015538671 for tool add -----

Reading it from the top:

  • the kind and the tool say what would happen. The expiry line appears only when the approval has one. An unread line appears when the stored row could not be read whole; approve and reject read it again in full before anything is decided.
  • the arguments are the exact tool call, one field per line, each with the byte size of its value as stored. A string over 2000 bytes (view.MaxFieldBytes) is shown by its head and tail with the number of bytes omitted between them, so a padded argument cannot push the rest off screen.
  • the presentation is what the policy meant a human to read, printed the same way, with a header saying it comes from the consumer's policy and may contain model-chosen text. It is part of the approval's hash, so what is on screen is what is bound.
  • the decide with line is the command with the database path, the run id, and the approval id filled in, because an operator who has to retype an id will mistype one.
  • the closing line is written by agentrt itself and names the approval and the tool. Everything above it except the labels may contain text the model or a server chose; this line cannot, and it is the last thing an operator reads before deciding.

Then the steps table: index, tool, decision kind, step status, policy outcome, observation kind, and the observation summary clipped to 60 characters.

STEP  TOOL  DECISION   STATUS             POLICY            OBSERVATION  SUMMARY
0     add   tool_call  awaiting_approval  require_approval  -

-limit (default 500) and -offset page the steps and the pending approvals together, and the output says how many more of each there are. -json returns the run summary, the page of step summaries with steps_total, the page of approvals with approvals_total, the waiting approval separately when there is exactly one, and the limit and offset; every approval in it is bounded by view.BoundApproval.

events <run>

agentrt -db runs.db events <run> [-limit n] [-offset n]

The run's audit log. The run is fetched first, so an unknown id is an error rather than an empty log. The stored events are then replayed through the same observers that print a live run, which means the format an operator reads afterwards is the format they watched: trace.Writer by default, trace.JSONL under -json.

2026-10-01 21:59:42.354 -0500  step.decided           {"kind":"tool_call","tool":"add","args":{"n":5},"reason":"add five"}
2026-10-01 21:59:42.354 -0500  step.policy            {"outcome":"require_approval","reason":"changing the counter needs an operator",...}
2026-10-01 21:59:42.354 -0500  approval.requested     {"approval_id":"7242f857...","capability":{"tool":"add"},"hash":"bc4c923b...",...}

-limit defaults to 500; -limit 0 streams the whole log a page at a time, printing each page as it is read rather than holding it. In text, a payload over 8000 bytes is cut with a count of what was cut. Under -json the payload passes through as stored, except that the store replaces one over MaxPageText (64 Ki characters) with {"truncated":true,"length":n,"head":...}, and when more events remain a final line {"meta":true,"truncated":true,"total":n,"returned":n,"offset":n} says so, marked meta so that a reader expecting one event per line does not mistake it for one.

approve <run> and reject <run>

agentrt -db runs.db approve <run>                                   # at a terminal: shows it, then asks
agentrt -db runs.db approve <run> -approval <id> -by joey -note "checked the diff"
agentrt -db runs.db reject  <run> -yes -note "wrong base"
Flag Default
-approval id the run's only pending approval (see the selection rule below)
-by who $USER; refused if both are empty
-note text none
-yes off

What happens, in order:

  1. The approval is selected by view.PendingApproval: a named one must be pending; without a name the run must have exactly one, and otherwise the error names up to 20 candidates and how many more there are.
  2. It is printed, as show prints it, in text mode.
  3. Consent is required. One of: -approval <id> (naming the approval is itself the deliberate act), -yes, or y or yes typed at the prompt approve approval <id> of run <run>? [y/N], which appears only when stdin is a terminal and output is text. Anything else typed declines, at exit 1. With none of the three available, as in a script, the command refuses at exit 2 and tells you to pass -approval <id> or -yes. Standard input counts as a terminal only when it is a character device that is not /dev/null, compared by identity rather than by path, so approve </dev/null refuses at exit 2 instead of prompting and declining; and reaching the end of input at a real terminal before anything was typed refuses the same way, rather than being recorded as a typed "n". (Both are v0.3.1 fixes; v0.3.0 prompted, read nothing, and declined at exit 1.) Under -json there is no prompt, so a decision needs -approval or -yes.
  4. The decision is bound to what was shown. The command passes the hash of the approval it read and printed to ApproveShown or RejectShown, which compare it with the stored hash inside the transaction that records the decision, with the update conditional on it. If anything changed the approval in between, a concurrent writer, a re-pause, a damaged row, the decision is refused, and the command exits 1 with approval <id> changed after it was shown; run the command again and read it before deciding. The hash is also recomputed from the stored fields, so a tampered row is refused.
  5. The event is echoed to stderr through trace.Writer, and the new statuses are printed:
approval 7242f85737a3600a21d31fe015538671: approved by joey
run 164ba4c9245839941e291b3d6e71bac5: WAITING_FOR_APPROVAL
the run stays waiting until the consumer resumes it

That last line is the most important in the tool, because approving is not resuming: the run is still WAITING_FOR_APPROVAL and nothing has executed. reject cancels the run with reason approval_rejected. A trailing argument after the run id is a usage error. -json prints the run summary and the decided approval, bounded, with the hash that was decided.

-by is a label. It is recorded exactly as given and never verified. It says who the operator says they are; it is worth as much as your trust in whoever can write the database.

cancel <run>

agentrt -db runs.db cancel <run> -by joey -note "no longer needed"

Ends any non-terminal run as operator_cancelled, with -by (default $USER) and -note, and no confirmation step. A terminal run is refused. Cancel needs no lease: a loop still executing the run stops at its next write, and a tool it had already started has its outcome appended as a late step.tool_finished event. This is the way to close a run that was approved and never resumed, since an approval cannot be un-approved; see Approvals and Resume.

verify

agentrt -db runs.db verify [-from N] [-to N] [-head SEQ:HASH]

Checks the hash chain over every run's events and changes nothing. It opens the database read-only and reads a page at a time. -from and -to bound the range by seq, and -to past the last event fails. It prints the chain's head, then either "intact" with the seq and hash of the last event checked, or the first event that does not chain with the hash expected and the hash found:

head: seq 20 27e4a4f4...
intact: 20 event(s), seq 1 to 20
last checked: seq 20 27e4a4f4...
head: seq 20 27e4a4f4...
broken at seq 15: the stored hash is not the one the event's fields and the previous event's hash give
  expected 5e5b76ea...
  found    236bb02f...
error: the event chain is broken at seq 15

The command prints the whole hashes; they are shortened here. With -json the intact result is an object with the head, the range, the last hash, the count checked, and "intact": true.

-head SEQ:HASH is a head kept elsewhere: from an earlier verify's "last checked" line, or from export.Follower.Head (Exporting Events). It fails, at exit 1, unless the walk reaches that seq with no break and the event there still has that hash. Without it, verify cannot see a chain cut at its end, because a shorter chain is still a chain (TestVerify_CutDatabaseFailsAKeptHead).

What an intact result means. The events are unaltered only up to a head kept where whoever can write the database cannot reach. Such a writer can recompute every hash after a row they changed, or cut events from the end, and the chain still verifies; keep the last checked seq and hash, and pass them back as -head next time. The chain covers events only, not the state tables, and an event stored before migration 6 carries no claim beyond the backfill. Re-checking a Run says this in full.

Exit 0 is an intact chain, and 1 a break, a -to not reached, or a -head not reached or not matched. Exit 2 is bad arguments, which are refused before the database is opened: a -head that is not SEQ:HASH, a -head seq past -to or before -from, a negative number, or an unexpected argument (TestVerify_IntactChainExitsZero, TestVerify_BrokenChainExitsOne, TestVerify_BadArgumentsAreUsage).

Why there is no resume

Resuming executes the approved request and then continues the loop, and continuing the loop means calling the agent, which needs the consumer's agent, its tools, its policy, and its model. A generic command cannot have those. It could only pretend to, and a resume that ran with a different tool set or a different policy than the run started with would be the one operation in the system that quietly changed the rules. So the runtime's own command stops at the decision, and the help text says where resume lives; a test asserts that it says so.

A consumer's resume is one line inside the binary it already builds:

driver, err := agentrt.NewDriver(agentrt.Config{Store: store, Agent: myAgent, Policy: myPolicy, Tools: myTools, Model: myModel})
run, err := driver.Resume(ctx, runID)

examples/live is the smallest worked example: -resume <run> rebuilds the same driver and calls Resume. The operator's sequence is therefore three commands, two of them the consumer's:

go run ./examples/live                              # pauses, prints the two below
go run ./cmd/agentrt -db <path> approve <run>       # the generic decision
go run ./examples/live -db <path> -resume <run>     # the consumer continues

The view package

The command is a thin renderer over view, which is where the read models live. Nothing in it decides anything except through the core's own functions.

Type or function What it is
RunSummary one run as an operator needs it: id, goal, status, reason and detail, step count, timestamps, model calls, usage, estimated cost, and PendingApprovalID. The model totals come from the run row, so a summary costs no extra query
StepSummary one step: index, id, tool, decision kind, step status, policy outcome, observation kind, and summary
Runs, Summary, Steps, Detail full reads: every run newest first, one run, a run's steps, a run with its steps and approvals
RunsPage, DetailPage, RunDetailPage the same, read in bounded pages through the store's page reads, with totals; what agentrt uses
PendingApproval the approval an operator means
ApproveShown, RejectShown, Approve, Reject, Cancel the core's operator paths, with errors wrapped so ErrRunNotWaiting, ErrApprovalDecided, and ErrDatabaseLocked match under errors.Is
BoundValue, BoundRaw, BoundApproval, MaxFieldBytes the 2000-byte head-and-tail bound on model- and policy-chosen strings, for text and JSON alike

PendingApprovalID is set only for a run in WAITING_FOR_APPROVAL, which is the only state in which an approval can still be decided, and only when exactly one is pending.

The exactly-one rule. PendingApproval(ctx, store, runID, approvalID): given an id it fetches that approval and requires it to be pending, so a decided or expired one is an error (view.ErrNotPending) rather than a second decision. Given no id, the run must have exactly one pending approval. None is ErrNoPendingApproval. More than one is ErrAmbiguousApproval, whose message names the candidates, up to 20, because choosing which of two authorizations to grant is a human's decision and not something a default should make on their behalf. That is why -approval exists at all: the common case needs it never, and the ambiguous case must not be guessed.

A consumer building its own surface, a web view, a chat bot, a steward runs show, uses these types rather than reading tables, and ApproveShown rather than Approve, so every surface shows an operator the same thing and binds the decision to it. Since v0.3.1 the approver package is the interface such a surface decides through, and approver/webhook is a signed HTTP channel over it; see Approval Channels.

The trace package

Two observers, for the two places events go, and the sanitiser both the command and the observers use.

trace.Writer(w) prints one aligned line per event: the local date, time to milliseconds, and zone (2006-01-02 15:04:05.000 -0700), two spaces, the event type padded to 22 columns, then the payload as recorded, passed through Sanitize. It is the format both consumers print to stderr under -trace, and what events prints. The payload is whatever the runtime recorded for that event type: the decision itself for step.decided, the policy decision for step.policy, and the approval's kind, capability, presentation, and hash for approval.requested.

trace.JSONL(w) writes one JSON object per event with seq, run_id, step_id, at in RFC 3339 with nanoseconds, type, and payload. The payload passes through as recorded: HTML escaping is off, so a payload containing <, >, or & is not rewritten on its way out, and an empty payload becomes {} so every line has the same shape. An event whose payload will not encode is not dropped: it is written with the payload as a JSON string and "invalid_payload": true. This is the stream to point a collector at until the export module of the Roadmap exists.

WriterWithErr and JSONLWithErr are the same observers plus a function returning the first write error, to call after the run: an observer cannot return an error to the runtime, and an audit stream that silently stopped writing is worse than one that says so.

Both hold a mutex around the write, because the runtime delivers events from whichever goroutine committed them, and a half-written line in an audit trail is worse than a slow one. Either can be passed straight to Config.Observer:

driver, err := agentrt.NewDriver(agentrt.Config{..., Observer: trace.Writer(os.Stderr)})

trace.Sanitize(s) is exported for any consumer that prints text a model or a server chose. JSON output is never passed through it.

Next

Persistence and Inspection is the data this command reads. Approvals and Resume is what a decision means. Security Model is what an operator can rely on, by attacker.

⚠️ **GitHub.com Fallback** ⚠️