Running the Issue Pipeline - axolotl-logic/project-buddy GitHub Wiki

A pi-subagents workflow that turns GitHub issues from the users on its config's allowedUsers list into merged pull requests. Each run is one sweep: it claims one issue, the oldest waiting, and takes it as far as it can. A lane is one pi session running the sweep on a schedule (see Schedules). Each lane works on one issue at a time, so the number of issues in progress is the number of lanes; no setting caps it. Lanes race for issues with claim comments on GitHub (see Claims). The Orchestrator instead runs one stage of an issue at a time as a stage job, several issues at once, and supersedes lanes. Its goal, rules and design decisions are on the Issue pipeline page, and its labels on Agent labels. The code lives in the repo's apps/issue-pipeline/ and runs from the copy that mise //apps/issue-pipeline:install puts in $ISSUE_PIPELINE_ROOT (see Change the pipeline). Paths below are relative to $ISSUE_PIPELINE_ROOT (default ~/.pi/agent/issue-pipeline) unless they start with $PI_CODING_AGENT_DIR (default ~/.pi/agent).

1. Pick the one GitHub account that does everything: opens PRs, posts every agent's review, and merges. It can be your own. Set it as account. Each sweep checks that gh is logged in as that account and stops if not.

2. Optionally create a fine-grained token for it, limited to the repo, and save it to secrets/<owner>__<repo>.token (chmod 600). Set tokenFile to that path. Without tokenFile, ops.mjs uses gh's keyring login.

3. Protect the base branch with the CI checks the pipeline should wait on. Don't require an approving review: GitHub never counts a PR author's approval, and here the author is the reviewer. If the repo needs that rule for people, either let the account bypass it or set mergeAdmin: true so the merge uses --admin.

4. Copy repos/example.json to repos/<owner>__<repo>.json and fill it in. commands.test and allowedUsers are required; the pipeline refuses repos without them. allowedUsers lists the GitHub usernames whose issues it takes and whose comments its agents read; each must also be a collaborator, and the account is always allowed. commands.setup runs in a new worktree; commands.teardown, if set, runs in runs/<repo>/issue-<n>/ when the worktree is removed, after a merge or split or by the end-of-sweep cleanup, to free what setup made elsewhere (project-buddy drops its pb_issue_<n> test database and removes its pb-issue-<n>-neon-proxy container).

5. Run one sweep by hand from a pi session and read the result:

   subagent({ workflow: "./workflow.js", cwd: "$ISSUE_PIPELINE_ROOT",
              args: { config: "$ISSUE_PIPELINE_ROOT/repos/<owner>__<repo>.json" },
              mission: false, maxSubagentSpawnsPerRun: 64 })
   

6. Start it with /pipeline-start [<owner>__<repo>] ($PI_CODING_AGENT_DIR/prompts/pipeline-start.md) in a pi session kept for the pipeline. The pipeline never starts on its own: only that session runs it. See Schedules.

Each issue gets branch agent/issue-<n> and one PR. The PR opens as a draft as soon as triage says ready: start-pr pushes an empty first commit ("Start work on #N"; the squash merge drops it) and opens the draft titled " (#N)" with a body that starts "Closes #N". That links the PR to the issue's Development section and closes the issue on merge, and the "(#N)" carries the issue number into the squash commit. After implementation, open-pr pushes the code, replaces the title and body with the implementer's (keeping both links), and marks the PR ready for review. A change with no code closes the draft instead, and a later fix that adds code reopens it (see Flows).

After triage says ready and the draft PR opens, the spec writer reads the issue, the project's coding practices, the wiki, the code, and the web where the repo can't answer, and writes a spec: Behaviour, Out of scope, Design, Tests, Wiki updates, Assumptions and risks, Sources. It reads each file in the config's practices list in full (default AGENTS.md and CONTRIBUTING.md; project-buddy adds wiki/Decisions-Code-and-tooling.md), and the spec must follow them or name each rule it can't and why. The implementer, both fixers and the first reviewers read the same files (decision spec-reads-practices). It posts the spec on the issue as a signed comment. The implementer builds to it, and the acceptance check fails a change that misses one of its Behaviours or Wiki updates. The review lenses, the fixer and the re-reviewers get the spec too: a finding must not ask for a change the spec forbids, and the fixer disputes a finding that can only be met by breaking the spec, quoting the spec line. A spec writer that needs an answer asks on the issue (agent:needs-info), and the answered issue goes back through triage, or to review if its branch already holds work.

With wiki: true in the config (or { "dir", "branch", "url" }), each issue's worktree gets the repo's GitHub wiki checked out at wiki/, on branch agent/issue-<n> of a shared clone in clones/<slug>.wiki, and hidden from the code repo. Wiki commits are reviewed with the code, since prepare-review appends them to the diff, are kept on the wiki repo's agent/issue-<n> branch at each push, so another lane or machine can carry on from them, and land on the wiki's main branch after the PR merges. If that push conflicts, the commits are saved as runs/<slug>/issue-<n>/wiki-unpushed.patch and the issue gets a comment. A repo whose wiki has no pages yet is skipped: GitHub creates the wiki repo only when its first page is made in the browser.

Each change runs one of four flows, by what it changed (#217). push, open-pr, a clean sync-base and finish-sync report codeChanged (whether the branch changes any file against the base) and wikiAhead (the wiki checkout's commits of its own), and open-pr, fix, the CI fix, sync and finish-sync save them as the issue's flows (src/workflow/flows.ts, acted on in stages/flow-step.ts).

A wiki-only change is still reviewed, fixed, synced and accepted: prepare-review puts its wiki diff in the review. If a later fix or CI fix adds code, open-pr reopens the closed PR (reopen), fills in its title and body and marks it ready, and the issue goes on as code and wiki. push pushes nothing when neither changed. finish-wiki publishes the wiki commits, closes the issue as completed with a signed merger comment that lists them, labels it agent:merged, clears its stop labels and claims, deletes the code branch, saves the state as done and removes the worktree. A publish conflict goes to the publish stage, which settles it with the conflict resolver and ends the issue through the same finish.

Every sweep ends with the cleanup op, which uses no model (#174). It runs after the issue's release and queued comments, and also when the sweep had no issue. It looks at each issue-<n> directory under work/<slug>/ and frees that issue's worktree, wiki worktree, both local agent/issue-<n> branches and commands.teardown resources when every check passes. The checks run in this order and stop at the first that fails:

1. It isn't the sweep's own issue.
2. GitHub says the issue is closed. Open issues, including stopped ones, are never touched, so a restart continues from the worktree. An unreadable answer also keeps it.
3. No lane holds a live claim on it.
4. Neither the worktree nor its wiki checkout has uncommitted changes.
5. Every local commit is on GitHub: on the base branch, the issue's remote branch, or its PR's head (refs/pull/<pr>/head), which keeps a squash-merged PR's commits after its branch is deleted.
6. The wiki checkout has no commit missing from the wiki's branch, so a saved wiki-unpushed.patch keeps it.

Each issue is checked and removed while holding <clone>.lock, one issue at a time, so two lanes never clean up the same issue at once. A check that fails or errors keeps the worktree: nothing is removed when there is doubt, and one issue's error doesn't stop the others. Run logs in runs/ and remote branches stay.

The log is the op's reply, { worktrees: [{ issue, outcome: "removed" }, { issue, outcome: "kept", why }] }, which the sweep's summary repeats as cleanup.worktrees, with cleanup.error set when the op itself failed. A worktree kept because of local work that is truly unwanted must be removed by hand: git -C clones/<slug> worktree remove --force work/<slug>/issue-<n>, delete the agent/issue-<n> branches in both clones, and run the teardown command in runs/<slug>/issue-<n>/.

One account posts everything, so every pipeline comment starts with a signature:

**🤖 issue-pipeline · reviewer (scope)**
<sub>agent `issue-pipeline-reader` · model `claude-sonnet-5-5` · run `<run id>`</sub>

Roles are triage, spec writer, implementer, reviewer (correctness / tests / scope), aggregator, fixer (round n), re-reviewer (lens, round n), check fixer, check-fix reviewer, acceptance verifier, and the steps that run no model: local checks, CI, sync, merger and coordinator (stops and parent closing). The run id finds the session with subagent({ action: "status", id }). Agent commits end with a Pipeline-Agent: trailer.

A hidden <!-- agent-pipeline:comment role=... --> marker in the signature is how the pipeline tells its own comments apart. A comment from the same account without the marker counts as a person's: if you reply to the pipeline's questions from that account, it counts as an answer.

The pipeline runs only when its operator starts it (Autumn, 2026-10-14). /pipeline-start creates one session-only schedule (sessionOnly: true, every 2m, overlap: skip, catchUp: none, quiet) named issue-pipeline-<slug>-<lane>, where <lane> is the first 8 characters of the session id. Every fire stays in that session, and no other pi session arms it. /pipeline-pause pauses it, and /pipeline-start in the same session resumes it. Closing the session stops the lane, and reopening it re-arms the schedule unless it was paused first. Run the pipeline in a session of its own, for example pi opened in $ISSUE_PIPELINE_ROOT, so its notices stay out of other work.

There are no project-wide schedules. Don't create one, since any pi session in that project would fire it.

For more lanes, open more pi sessions and run /pipeline-start in each. They can share one directory: each schedule is bound to its own session. Lanes coordinate through claim comments on GitHub, not local files, so they may also run from separate copies of the pipeline directory or on other machines. A lane on a fresh copy rebuilds an issue's worktree from its PR branch. Lanes sharing a directory share its clones and worktrees, which saves disk and setup time. The one local-only guard is the parent reservation: two lanes in separate directories may both close the same finished parent issue, which posts the closing comment twice.

A lane takes a new issue like this (raceForClaim in src/claims.ts):

1. The sweep lists up to claimCandidates (default 5) waiting issues, oldest first, skipping issues another lane holds.
2. The lane posts a claim comment on the first candidate (<!-- agent-pipeline:claim lane=<lane> sweep=<sweep> -->) and adds agent:working.
3. It waits a random time within claimWaitSeconds (default [5, 20]), so that every racing lane's claim has landed.
4. It lists the issue's live claims. If its own claim is the earliest (lowest comment id), it proceeds. Otherwise it edits its claim to claim-closed with an "Abandoned" note naming the winning lane, and tries the next candidate. A lane that sees a live claim before posting skips the issue without commenting.

The winning claim stays live while its lane works the issue: each sweep that ends with the issue unfinished keeps the lease on that lane and edits the claim, so the lane's next sweep resumes the issue first and other lanes skip it. A claim closes when the issue leaves the pipeline (merged, stopped, asking questions, split). A claim or lease left by a lane that went away expires after leaseHours (default 3) without an edit; then any lane may take the issue.

The wait must be longer than the time GitHub takes to list a new comment, which is normally well under a second. If it weren't, two lanes could each fail to see the other's claim. The pipeline's own comments skip claim comments when they build an agent's issue context.

/pipeline-start passes the lane id in args.lane. A workflow run without it is a lane of its own, so it can't resume an issue an earlier run of it left unfinished until the lease expires.

Schedules copy the script when created. After editing workflow.js, run /pipeline-pause, delete the schedule (subagent({ action: "schedule.delete", id })) after its sweep finishes, and run /pipeline-start again. Edits to ops.mjs, the agents and the configs take effect on the next sweep.

Change apps/issue-pipeline/ through a PR into integration, like any code; mise //apps/issue-pipeline:check typechecks it, lints it, builds it and runs its tests. The sources are in src/, and mise //apps/issue-pipeline:build builds them with esbuild into dist/ (#165). The workflow's modules are strict TS in src/workflow/, which the build bundles into dist/workflow.js ahead of the sweep body (#177). Every stage runs on a step machine, because pi-subagents rejects nested async functions in a workflow: each stage is a sync next in src/workflow/stages/ that takes the issue's JSON-safe state and the results of its last launches and returns its next launches or done, with the comments and the stop to apply. The sweep (its entry, the claim, the stage groups, closing parents, the release and the summary) is a strict-TS driver in src/workflow/sweep.ts, and workflow.js only awaits the launches the driver asks for (#190, #223). Merging changes nothing that runs. To run it, check out the merged commit, run npm ci at the repo root (the build needs esbuild), and run mise //apps/issue-pipeline:install:dry-run, then mise //apps/issue-pipeline:install, between sweeps or with the lanes paused (/pipeline-pause). The install first builds src/ into dist/, stops with install: build failed if the build fails, and installs the built workflow.js and ops.mjs; a dry run writes only dist/. Each op reads ops.mjs fresh, but a schedule keeps the workflow.js text it was created with, so recreate the lanes after a change to workflow.js. The install backs up each file it replaces to $ISSUE_PIPELINE_ROOT-backups/<timestamp>-install/; to roll back, install an older commit or copy the backup back. Don't edit the installed files: the next install overwrites them. Per-repo configs, setup and teardown scripts and secrets/ are local and are edited in place.

Set ISSUE_PIPELINE_ROOT to a scratch directory, put a fake gh first on PATH, and point cloneUrl in a scratch config at a local bare repo. Then pipe requests in:

printf 'BEGIN_OPS_REQUEST\n{"op":"verify","config":"/tmp/cfg.json","number":7}\nEND_OPS_REQUEST\n' | node ops.mjs

A stop means the issue failed a control: a limit or check it didn't pass. The issue and its PR get agent:stopped-<control> and a signed coordinator comment naming the control, what it checks, the reason, details and how to resume. Agent labels lists every label, what triggers it and what to do. The controls are the STOPS table in src/stops.ts; a new stop reason in workflow.js needs a matching row there and on Agent labels.

A network outage doesn't stop an issue (#204). When one of these ops fails with a network error, the stage ends for the sweep with the issue still at it, and the next sweep runs the stage again:

• prepare-review at the start of acceptance and of the security review
• sync-base in sync
• verify and ci-status in checks
• merge in merge

A network error is a DNS failure (Could not resolve host), error connecting to api.github.com, Connection timed out, Connection reset, or an HTTP 502, 503 or 504 from gh or git (isNetworkError in src/workflow/stages/network.ts). The issue's log gets deferred: network error during <op>; will retry next sweep, and the deferral charges no tokens: acceptance and the security review charge their session only once the diff is ready. Every other op error still stops the issue as before, and so does a network error in any other op, such as prepare-review in a review round, push in a fix, finish-sync or open-pr, where a paid session has already run.

Every op that pushes the PR branch (start-pr, open-pr, push, sync-base and finish-sync) first records the head it is pushing in runs/<slug>/issue-N/pushed.json, as { "sha", "op", "at" }, replacing the last record. A sync-base whose push fails puts the worktree back at its head before the rebase. If a push lands but the state save after it fails, the next claim finds the branch's head differs from the saved one. When that head equals the record and the record's op is a sync with the base (sync-base or finish-sync), the claim adopts it: it sets the head, clears the passed head, logs adopted the pipeline's own push <sha8>; its state save had failed and continues at sync, so the head goes through checks, acceptance and the security review again. The claim adopts only a sync's push, and only from a state saved at sync or later (sync, checks, accept, security or merge): any other push is followed by a review, the code review after open-pr or a fix's push, or the check-fix review after a CI fix's push at checks, and sent to sync it would skip that review. Such a head, any other head, or a missing or unreadable record still stops the issue with agent:stopped-branch-changed. The record is kept only on the machine that pushed, so a lane on another machine still stops (see Agent labels).

No file is off limits: agents may edit CI workflows, auth, dependencies, migrations and config when the issue needs it, and no prompt or reviewer judges a change by the kind of file it touches. The only content-based check is the security review, the last review round. After acceptance passes, a security reviewer reads the full final diff and reports only changes that would harm users (user-harm), violate their rights (rights), cause financial harm to users or the operator (financial-harm), or introduce a risk of abuse (abuse-risk). Its findings go through the usual fix rounds with fresh round counts; the fixed branch then goes back through sync, checks, acceptance and the security review. A security finding still open when the fix rounds run out, or found again after maxFixRounds security reviews, stops the issue with agent:stopped-<category>.

To take one issue right away, launch one run for it:

subagent({ workflow: "$ISSUE_PIPELINE_ROOT/workflow.js", async: true,
  args: { config: "<abs path to repos/<owner>__<repo>.json>", issue: 19 } })

The run takes only that issue, but refuses an issue another live sweep holds.

A named run, and so a pinned schedule, leaves an issue with an agent:stopped-* label alone: it claims nothing, and its summary reports the issue as left-stopped, with the label as its reason. This keeps a pin from redoing a stopped issue every interval, as it did for #236 (#251). For a deliberate one-shot on a stopped issue, add takeStopped: true to args; its claim clears the label. Removing the label resumes the issue on the next run.

To run an issue only up to a stage, add through: "<stage>" (#210). The run begins no stage after that one, then releases the issue at its next stage with its state saved, and a later sweep continues from there. For example, { issue: 19, through: "spec" } triages #19, opens its draft PR, writes its spec and stops at implement. The stages, in order, are triage, spec, implement, open-pr, review, fix, sync, checks, accept, security and merge. A pinned run doesn't check the issue's after marks.

A stage job runs one stage of one issue outside pi-subagents (#232, #233):

node $ISSUE_PIPELINE_ROOT/job.mjs --config <repos/<owner>__<repo>.json> --issue N --stage <stage> [--lane L] [--resume] [--stack-on P]

It claims #N, runs only that stage, releases the issue and prints the sweep's summary as one JSON line. With --stack-on P (an issue other than N), an issue with no branch of its own starts on #P's branch; see "Stacked branches" below. A job for an issue with an agent:stopped-* label claims nothing and prints a summary that reports the issue as left-stopped (#251); remove the label first. Ops run the installed ops.mjs; each model session runs pi --mode rpc as its installed agent, with its JSONL in runs/<slug>/issue-N/sessions/<launch key>.jsonl.

job.json. While a job runs, runs/<slug>/issue-N/job.json holds its pid, issue, stage, lane, startedAt, the steer socket and its outcome (null while it runs), and one record per launch it started: key, kind (session or ops), pid, session (the JSONL path, null for ops), startedAt, lastEventAt, toolFailures (consecutive failed calls of the tool that failed last; a success, or another tool's failure, starts the count again), lastToolFailure (that last failure's tool and error text, clipped to 2 000 characters; null after a success), usage (summed from its assistant messages), tokenBudget (the session's budget, min(roleBudgets[role], sessionTokenCap); null for ops and in an older file), contextTokens (the token total of its latest assistant message that reported usage, its live context size; 0 before the first, and in an older file), peakContext (the highest turn context the session has shown, a turn's context being the input, cache-read and cache-write tokens of one assistant message; 0 for ops and for a session with no usage yet, and for a launch in an older file), contextStop ({"context":N,"result":"handoff"|"error"} when the session passed contextLimits.hard at turn context N and was stopped, null otherwise), resumed and outcome (null until its result goes back to the sweep, then ok or failed). A session's retry after a stop at contextLimits.hard (#272) is its own record, keyed <key>.retry, with its own session JSONL, usage counted from zero, peakContext, contextStop and tokenBudget; when the launch's result goes back to the sweep, both records get the same outcome. The job rewrites it atomically (a temp file renamed over it) on every record a session prints and whenever a launch starts or ends. When the job ends, outcome is {"kind":"finished"}, {"kind":"stopped","signal":…} or {"kind":"failed","error":…}, with endedAt. When the job removes its own checkout at exit (see worktrees-and-branches), lostChanges lists the uncommitted paths it dropped (wiki paths under the wiki's folder), [] when there were none; a writer's exit push appears as the launch exit-push-1. A job whose file can't be written runs on and says why on stderr.

Stop. kill <pid> (SIGTERM) or Ctrl-C (SIGINT) stops a job. It starts no more launches, sends abort to each live session, kills any still running after 5 s, and lets ops already running finish. If it holds the claimed, open issue during its stages, it saves the issue's state at its current stage with no lease (save-state, never release, which would renew the lease), so the next job for that stage starts from the saved state. It then exits 128 + the signal (143 or 130) with nothing on stdout. A claim in flight when the stop comes is left to expire after leaseHours.

Resume. Run the same stage with --resume to carry on a stopped job's sessions. A session launch with no outcome in the earlier job.json whose file is still on disk reopens with pi --session <that file> and a prompt saying the session was interrupted; its tokens are its recorded usage plus the new process's, so each counts once. Any other launch starts fresh, as does every launch when there is no readable earlier job.json. Ops launches are never resumed.

Steer. Send a message to a live session of a running job:

node $ISSUE_PIPELINE_ROOT/job.mjs --config <…> --issue N --steer <launch key> --message "<text>"

The job listens on the socket job.json names (runs/<slug>/issue-N/job.sock, or a hashed name in the temp folder when that path is too long for a socket) and passes the message to pi as an RPC steer. The command exits 0 when pi accepted it, and 1 with the reason when there is no running job, the key is not a live session, or pi refused.

Boundary. Print what a stage job would run with, starting nothing (no claim, ops, pi, job.json or socket):

node $ISSUE_PIPELINE_ROOT/job.mjs --config <…> --issue N --stage <stage> --print-boundary

It prints one JSON line, {"role","checkout","env","network"}, where checkout is the job's own checkout, work/<slug>/jobs/issue-N-<stage> (not created by the print), env lists the names, never the values, of the environment its role's sessions get, and exits 0. It can't be combined with --steer. The repo config's optional jobRoles key sets each role's (reader, writer, merger) tokenEnv, the job variable whose value that role's sessions get as GH_TOKEN, and network, the hosts it may reach (recorded only), for example "jobRoles": {"writer": {"tokenEnv": "PIPELINE_WRITER_TOKEN"}}. A malformed jobRoles fails the job with exit 1. See Job boundaries.

The orchestrator (#234) starts stage jobs itself, as each issue needs them and as their files allow (decision stage-jobs-replace-lanes). Run it as a long-lived process, under launchd or in a tmux window, not inside a pi session:

node $ISSUE_PIPELINE_ROOT/orchestrate.mjs --config <repos/<owner>__<repo>.json> [--once]

Each tick it reads the open issues through the sweep op's own code (openIssues and classify, so stop labels, skipLabels, allowedUsers, takeNewWork and after marks count as they do for a sweep), each issue's saved state and its job.json. It decides, starts the jobs it chose and sleeps orchestratorIntervalSeconds (default 60), until SIGTERM or SIGINT. A tick that fails says why on stderr, and the next one runs at the next interval. --once runs one tick and exits 0, or 1 when the tick failed, for tests and cron. A bad flag or no --config exits 1 and names the problem.

Its config keys, shown in repos/example.json, are maxJobs, stageCaps, tokensPerHour and orchestratorIntervalSeconds, the watchdog's maxResumes, silentMinutes and failureStreak, and the escalation agent's escalationAgent (default null, off), escalationAgentActs (default false, log only), escalationAgentTimeoutSeconds (default 300) and escalationAgentModel (default anthropic/claude-sonnet-5-5), with the defaults below. A config without the escalation agent's keys runs no agent.

What runs. An issue runs the stage its state names, or triage with no state or with answered questions. An issue at stopped or done, or whose job is live, gets no decision. A job is live while job.json has no outcome and its pid runs, or while the job the orchestrator started for the issue runs (before it has written job.json). Within a tick, issues at a later stage go first, then older issues.

• Readers (triage, spec, review, accept, security, and open-pr) start whenever an issue is at that stage, up to maxJobs live jobs in all (default 4) and the stage's cap in stageCaps (default 2 for each reader stage; open-pr, which changes no files, has none).
• Writers are gated on files. The spec writer records the repo paths or globs the change will touch as st.files; an issue without them takes the whole repo (**). The live implementations are the issues past implement until they merge or stop, and the issues at implement whose job is live. An implement job waits while its files overlap any live implementation's. A fix, sync (the conflict resolver) or checks job of an issue already live waits only for an older overlapping one (earlier createdAt, then lower number), so two live issues never wait on each other. It doesn't wait for one that merges after it, either: when the issue is in the merge queue, an older overlapping issue that is queued later or not queued doesn't hold it, so the queue's first issue always moves and a writer can't wait on a merge that waits on it. A writer started in a tick counts as live for the rest of it.
• Overlap is conservative. Two globs overlap when they are equal, when one has no wildcard and the other matches it, or when both have wildcards and one's text before its first *, ? or [ starts the other's. So src/** overlaps src/a/b.ts, src/*.ts overlaps src/*.md (a false positive, accepted) and src/a.ts doesn't overlap docs/x.md.
• Merges go in the order of what they unblock. The issues at sync, checks, accept, security or merge rank by how many waiting issues (open, not stopped, not live, at implement or earlier) depend on them, through an after mark naming them or overlapping files; then the lower PR, then the lower issue number. Only the top one starts its merge job, only once it is at merge and no merge job is live. The job gets the ranked list as --merge-queue n1,n2,…, and ops.merge then waits (queued) only for an issue ahead of it in the list that is open, not stopped and at one of those stages, in place of the lowest-PR order (merge-order).
• Stacked branches (#290, decision stacked-branches). An issue that waits on its after mark, when the mark names exactly one open issue P, starts its triage, spec or implement stacked on P's branch once P is at checks or later with no open findings, unless P is stacked itself (one level only); until then it is held after:#P. A stacked issue runs its stages as usual, but at merge it is held stacked-on:#P while P is open, and isn't ranked for merge. When P's branch head moves past the head the issue was put on, or P merges (its PR merged, it is at publish, or it was closed as completed), a stacked issue at sync through merge is started at sync, which rebases it onto P's new head or, once P merged, onto the base. After a clean rebase an issue sent back from merge re-runs only the tests; one sent back from checks, accept or security goes through acceptance and the security review again. A stopped P (a stop label, or saved at stopped) holds its dependents after:#P:stopped, and a P closed without merging holds a stacked dependent after:#P:closed-unmerged; they are held, not stopped. The file-overlap gate never holds an issue for its prerequisite, nor a prerequisite for its dependent.
• Rate limit. runs/<slug>/orchestrator.json keeps, for each launch (its job's startedAt and launch key), the last token total its job.json showed and the increments since, with the times they were seen, so the totals survive a job.json being overwritten. Once the increments in the last hour reach tokensPerHour (default 2 000 000), every start is held, the watchdog's restarts included. The file also keeps the jobs the orchestrator started (issue, stage, pid, time), an hour of each.

Starts. Each start is a detached node $ISSUE_PIPELINE_ROOT/job.mjs --config <path> --issue N --stage S --lane orchestrator [--merge-queue …] [--stack-on P], its stdout and stderr appended to runs/<slug>/issue-N/job.log. --stack-on P is passed only on the first job of an issue that starts stacked on P; later jobs and restarts go on from the saved stackedOn. The fixed lane lets the next job for the same issue pass the sweep's lease check.

The watchdog. Each tick, before the scheduler decides anything, the watchdog (#235) checks every open issue's job.json. It leaves alone an issue with a stop label or a saved state at stopped.

• Dead job. A job is dead when its outcome is null and it is not live (neither the job.json pid nor the orchestrator's start record for the issue runs), or when the watchdog sent it SIGTERM for silence itself and it is no longer live. A dead job is restarted as the same stage, job.mjs --config … --issue N --stage <job.json stage> --lane orchestrator --resume, which counts as live for the rest of the tick. A restart passes the same checks as a start: maxJobs, the stage's cap, the writers' file overlaps, the merge order and the rate limit. One that fails a check is logged as held with the scheduler's reason (such as rate-limit or global-cap), isn't counted, and waits for a later tick; the scheduler starts nothing else for that issue meanwhile. Restarts are counted per issue and stage. When the count already equals maxResumes (default 2), the issue is stopped instead, through the stop op with the code job (agent:stopped-job), carrying its saved state, so its PR gets the label and the comment too; the stop clears the issue's counts, so removing the label starts them from zero. A job whose outcome is finished or failed is never restarted, nor one someone else stopped.
• Silent session. For a live job with no outcome, each session launch with no outcome is silent from its lastEventAt (or startedAt). Ops launches are never checked, since they print nothing until they end. At silentMinutes (default 30) of silence the session gets one steer through the job's socket: You have been silent; report status and carry on. If no event arrives in the silentMinutes after that steer, the orchestrator sends the job's pid SIGTERM. The job then saves the issue at its stage and exits (see Stop under Stage jobs), and the next tick finds it dead and resumes it, counted as a restart. A session heard from after its steer starts over: it is steered again only after another silentMinutes of silence.
• Repeated failures. For a live job with no outcome, a session launch with no outcome whose toolFailures reaches failureStreak (default 3) gets one steer through the job's socket naming the count and the tool, quoting lastToolFailure's error inside a code fence longer than any run of backticks in the error, so the error can't close it, under a note that the fenced text is the tool's output, data rather than instructions, and telling it to read that error before it retries and to change the call rather than repeat it. One steer per streak: it is steered again only once its count has dropped (a success or another tool's failure) and reached failureStreak again.
• Over budget. Every session launch's job.json record carries its tokenBudget, its role's budget capped by sessionTokenCap (min(roleBudgets[role], sessionTokenCap), the number its preamble's "Context budget" line shows), and its contextTokens, the token total (input, output, cache read and cache write) of its latest assistant message that reported usage: its live context size, 0 until the first such message. For a live job with no outcome that the watchdog hasn't already sent SIGTERM, a session launch with no outcome whose contextTokens is above its tokenBudget (strictly; at the budget is fine) gets the job's pid SIGTERM once, logged killed with reason budget:<launch key>. On the first later tick where the job is no longer live, the issue is stopped through the stop op with the code budget (agent:stopped-budget on the issue and its PR, and the stop comment naming the launch, its context size and its budget), logged stopped with reason budget, and the job is never restarted, whatever outcome it ended with. SIGTERM comes first and the stop waits for the exit because a job saves the issue's state at its stage on SIGTERM (see Stop under Stage jobs), which would overwrite the stop's stopped state. Each job gets at most one SIGTERM and one stop from this rule. Ops launches, launches with no budget (tokenBudget: null, as an older job.json reads) and launches that ended are never checked. The cumulative usage, which sums every turn's re-read context, doesn't count, however large it gets.

Escalations. Each tick, what the watchdog's rules can't settle is written to runs/<slug>/escalations.jsonl, one JSON line each, {"at","issue","stage","reason","label"}, and logged as escalated. An open issue escalates for a stop label the rules didn't expect (reason unexpected-stop: any stop except agent:stopped-job, which the rules end in, and the agent:stopped-budget the watchdog set itself for the issue's current job, the one with the same job.json startedAt; any other agent:stopped-budget, such as the sweep's own up-front budget check, escalates), for agent:stopped-acceptance (reason acceptance-fail) and for agent:needs-info (reason needs-info). stage is the stage its saved state names, or none. Each label escalates once while the issue carries it, and again if it is removed and comes back.

Escalation agent. Optional and off by default (#282). Set escalationAgent to an installed agent definition's name, such as the shipped "issue-pipeline-escalation" (agents/issue-pipeline-escalation.md, installed with the other agents), to turn it on. It runs only in a tick that wrote at least one new escalation, and then exactly once, however many there are; a tick with none, including a later tick for the same still-escalated label, runs no agent. It is one pi --mode json with the definition's system prompt and thinking, --model escalationAgentModel, -nc -ns, and only read-only tools: the definition's tools that are among read, grep, find and ls (all four if none are), plus structured_output. It gets no gh token variables (sandbox-ready-jobs), and keeps its session and reply schema in runs/<slug>/escalation-agent/. Its prompt quotes the new escalation lines as data and, for each escalated issue, names its job.json, job.log and sessions/ in runs/<slug>/issue-N/ and inlines its saved state as fenced data, since the state lives in a GitHub comment. It replies { recommendations: [{ issue, action }] }, where action is { kind: "steer", message }, { kind: "resume" }, { kind: "stop", code, reason } with a stop code, or { kind: "none" }. Recommendations for issues not escalated in this tick are ignored, the first one for an issue counts, and an escalated issue with none counts as none. Each escalated issue gets one recommended line, with the escalation's stage and reason steer, resume, stop:<code> or none.

By default that is all: nothing is steered, started, stopped, labelled or written. With escalationAgentActs: true, each recommendation is carried out through the watchdog's own commands; the agent never edits labels, files or state itself. A steer goes to the latest session launch with no outcome of the issue's running job, through its job.json socket, logged steered with reason agent:<launch key>, or is logged held with reason no-live-session. A resume restarts a job that isn't live as its job.json stage with --resume, through the same start gate as the watchdog's restarts, logged resumed with reason agent; it counts as live for the rest of the tick, and doesn't count towards maxResumes. It is logged held with the gate's reason, live or no-job (no job.json) instead. A stop runs the stop op with the code and reason and the issue's saved state, logged stopped with reason agent:<code>; like any stop, it labels the issue and its PR. A resume doesn't remove a stop label. A failed run doesn't stop the tick: each escalated issue gets an agent-failed line, reason no-definition (the definition is missing or malformed; no pi starts), timed-out (escalationAgentTimeoutSeconds passed), failed (pi exited non-zero, gave no structured output or couldn't start) or bad-reply (its reply didn't parse), and the tick goes on to save the watchdog's memory and start jobs. A command that throws while acting is logged agent-failed with reason failed. The run blocks the tick for at most its timeout. A steer's message comes from a model that read logs, which can carry injected text; that is why acting is off by default.

What the watchdog remembers between ticks (restart counts, steers, the jobs it sent SIGTERM for silence, the jobs it sent SIGTERM for their budget, by <issue> <job startedAt>, with the launch, its context size and budget and whether the issue is stopped yet, and the labels it escalated) is in runs/<slug>/watchdog.json. It is saved only once every action of the tick is done, so an action that failed (for example a stop whose gh call failed) is worked out again next tick.

The log. Every decision is one JSON line in runs/<slug>/orchestrator.log: {"at","issue","stage","action","reason"}, with action started (reason free, or merge-first for a merge) or held, with one of global-cap, stage-cap, rate-limit, files-overlap:#M, merge-order:#M, after:#M (the issue comes after open #M), after:#M:stopped (its prerequisite #M is stopped), after:#M:closed-unmerged (the issue it is stacked on was closed without merging), stacked-on:#M (a stacked issue's merge waits for open #M) or lane:<lane> (another lane holds a live lease on it). The watchdog's actions are lines of the same form: resumed (reason dead), stopped (reason max-resumes or budget), steered and killed (reason silent:<launch key>), killed (reason budget:<launch key>), steered (reason failures:<launch key>) and escalated (reason unexpected-stop, acceptance-fail or needs-info). The escalation agent's lines are recommended (reason steer, resume, stop:<code> or none) and agent-failed (reason no-definition, timed-out, failed or bad-reply), and, with acting on, steered (reason agent:<launch key>), resumed (reason agent), stopped (reason agent:<code>) and held (reason no-live-session, live, no-job or the start gate's reason). Read it with tail -f, or grep '"issue":12,' for one issue.

Lanes and /pipeline-start still work, and a lane's live lease holds the orchestrator off that issue (lane:<lane>). Run one or the other on a repo, not both, since a lane's sweep doesn't see the orchestrator's caps or file sets.

When rebasing onto the base branch conflicts, sync-base leaves the rebase stopped in the worktree and a writer (the conflict resolver) finishes it, keeping both sides' intent, then runs the type check and the touched unit tests. finish-sync checks that the rebase finished on the base sync-base rebased onto (its baseSha, which the sync stage passes on) with no conflict markers left, and pushes. If the base moved while the resolver worked, the resolution is still pushed, and the merge op then finds the branch behind and rebases it as usual (#279). The resolved head then goes through every check again, and acceptance reads the final diff; it is not re-reviewed. Conflicts don't stop the issue on their own: when sides contradict, the resolver keeps what the base now does and re-applies the branch's change on top, and if it can't finish the rebase at all, a fresh resolver tries next sweep. Only maxConflictFixes (default 2; project-buddy 100) and rebaseTokenBudget bound it. The resolver, and every session of the re-checks after a rebase the merge stage asked for (check fixes, acceptance and the security review), are charged to rebaseTokenBudget (default 500000), not to issueTokenBudget, and that allowance resets when the issue merges or restarts (#201). A resolver that doesn't fit it stops the issue with agent:stopped-budget; only used-up attempts or a rejected resolution stop it with agent:stopped-conflict. A resolver that runs out of room returns handoff, and a continuation resolver picks up the rebase still in progress from its handoff, without counting as another attempt; past maxContinuations (default 1 per issue) the rebase is aborted and the issue stops with agent:stopped-too-large (#269). A PR someone merged by hand is treated as merged at the next merge step.

When merge replies behind, the same sweep rebases the branch at once, re-runs only the tests, and tries the merge again (#205). This happens once per sweep: a second behind waits for the next sweep, and so does a conflict from merge, or a conflicting same-sweep rebase whose resolver doesn't fit under the spawn cap; the next sweep checks the resolver's budget and the lease again. Each rebase that changes the head still counts toward maxSyncs.

An issue can build on another issue's unmerged work (#268). The orchestrator passes --stack-on itself to an issue waiting on a prerequisite that has reached checks (#290, see Stacked branches under Orchestrator). job.mjs --stack-on P makes the claim of an issue with no agent/issue-N on GitHub start the worktree (and the job's checkout) at origin/agent/issue-P, and save stackedOn: {issue: P, headSha} in the state; a missing agent/issue-P fails the claim. An issue that already has a branch ignores --stack-on and keeps its saved stackedOn. While #P is open, sync-base rebases only the issue's own commits (git rebase --onto origin/agent/issue-P <saved headSha>) and saves the prerequisite's new head. Once #P's PR has merged, or #P is closed as completed, it rebases them onto origin/<base> instead and clears stackedOn, and the issue syncs as any other from then on. Conflicts take the resolver path above, with finish-sync checking the rebase against the prerequisite's head while stacked; the issue takes the new stackedOn only when the resolution is pushed. While stacked, prepare-review and verify measure from the prerequisite's head, so reviews, the changed files and fails-before-fix see only the issue's own change. Because of that, while #P is open the merge op replies queued, naming #P (#289), and the orchestrator holds the issue's merge stacked-on:#P; once #P has merged, merge replies behind until a sync has moved the issue onto the base. It never squashes #P's unreviewed commits into the base. Still measured against the base: the flows' codeChanged, push's ahead check and the conflict resolver's prompt.