CI Parallel Gate - developerz-ai/ultimate GitHub Wiki
bun run check (x build --target static --no-preflight && x verify) is the gate on a laptop: one
process tree, bounded to a memory budget. On CI, where wall time is what a pull request waits on,
the same gate splits across jobs β each step runnable on its own, the parallel test steps sharded β
and one aggregate job folds the parts back into the one verdict. Green means exactly what the local
gate's green means: every step, over the whole tree, passed.
x verify --only <step>[,<step>β¦] --json runs just those steps, in one process, paying nothing
for the others (no app load for a step that does not need one, no test run for a static step). Its
JSON is a part: x verify merge reads it back.
| Step | One job runs | Needs |
|---|---|---|
typecheck |
x verify --only typecheck --json |
nothing |
lint boundaries filesize package-shape errors
|
x verify --only lint,boundaries,filesize,package-shape,errors --json |
nothing |
unit |
x verify --only unit --shard i/n --json |
nothing (PGlite in process) |
contract |
x verify --only contract --shard i/n --json |
nothing |
job |
x verify --only job --shard i/n --json |
nothing |
live |
x verify --only live --json |
a Postgres with wal_level=logical (TEST_DATABASE_URL) β serial, never sharded |
e2e |
x verify --only e2e --json |
the static build (x build --target static --no-preflight), Chrome β serial, never sharded |
eval |
x verify --only eval --json |
nothing; baselines committed |
drift contract-diff
|
x verify --only drift,contract-diff --json |
nothing |
budgets seo i18n policy
|
x verify --only budgets,seo,i18n,policy --json |
the static build β budgets weighs .x/build-stats.json
|
manifest roadmap
|
x verify --only manifest,roadmap --json |
nothing |
The grouping is yours: any split works as long as every step lands in exactly one part (or, for a
sharded step, in shards 1..n exactly once). x verify merge refuses anything else.
x verify --only unit --shard 2/4 --json > part-unit-2.json- Only with
--only, and only overunit,contractandjob.liveande2eare serial by design (one replication slot, one built output) and are refused withX_VERIFY_SHARD_INVALID. - The split is a pure function of the step's sorted file list: round-robin by default, so job
2 of 4 is the same files on every runner and on your laptop.
--timings <file>(Bun's--timingsfile, e.g. the.x/test-timings.jsonevery parallel run refreshes β cache it between CI runs) splits greedy longest-first instead; a file the timings do not know is costed at their median. - The part's JSON carries
data.shard(index,total, and per step thecorpusHashof the whole list plus thefilesthis shard ran), and each step carries the same facts assteps[].shard. - A shard whose slice is empty, or ran only skipped tests, is not red: the zero-tests floor
(
x.verify.json) is applied bymerge, on the counts summed over every shard. - An app's coverage floor is not judged by a shard β one slice cannot hold it. The shard is green
on coverage and carries what it covered in
data.coverage.unit.facts;mergefolds every shard and judges once (X_COVERAGE_BELOW_FLOOR). Shards that carry nothing fold to 0%, never to a skip. - Under
--jsoneach finished step is one line on stderr ({"step":"unit","ok":true,"ms":61250}), so a cancelled job's log ends on the last step that finished. Redirect stdout only. - A red shard reproduces locally with the same flags.
x verify merge parts/*.json --jsonReads each part (the last JSON line of the file, so bun run check --json output works too) and
answers the gate:
- every step of the gate appears β
X_VERIFY_MERGE_INCOMPLETEnames a step no part ran; - a sharded step has shards
1..nexactly once, all with onecorpusHashβ a missing shard, a duplicate, or two corpora (jobs on different commits) is named; - and their
filespartition that corpus: a file in two shards, or in none (the union does not hash tocorpusHashβ shards split by two rules, one with--timingsand one without), is named; - the zero-tests floor is applied to each sharded step's summed counts;
- an app's coverage floor is applied to the fold of a sharded
unitstep'sdata.coverage.unit.facts; - a red step in any part is red here, with its findings and output kept.
The merged document has no notAGateRun β it is the gate's answer β and its durationMs is the
slowest part's, which is the wall time CI waited. A part that is not an x verify --json
document is X_VERIFY_MERGE_INPUT β and so is a step whose durationMs is not a number or whose
findings are not each a code, cause and fix.
name: ci
on: [push, pull_request]
jobs:
part:
name: ${{ matrix.name }}
runs-on: ubuntu-latest
strategy:
fail-fast: false # every part must upload, red or green: merge decides
matrix:
include:
- { name: typecheck, only: typecheck }
- { name: static, only: 'lint,boundaries,filesize,package-shape,errors' }
- { name: unit-1, only: unit, shard: 1/4 }
- { name: unit-2, only: unit, shard: 2/4 }
- { name: unit-3, only: unit, shard: 3/4 }
- { name: unit-4, only: unit, shard: 4/4 }
- { name: contract-1, only: contract, shard: 1/2 }
- { name: contract-2, only: contract, shard: 2/2 }
- { name: job, only: job, shard: 1/1 }
- { name: tail, only: 'eval,drift,contract-diff,manifest,roadmap' }
- { name: built, only: 'e2e,budgets,seo,i18n,policy', build: true }
- { name: live, only: live, postgres: true }
env:
TEST_DATABASE_URL: ${{ matrix.postgres && 'postgres://postgres:postgres@localhost:5432/postgres' || '' }}
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
- run: bun install --frozen-lockfile
- if: matrix.postgres # live needs logical replication: a service container cannot pass -c
run: |
docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:17 -c wal_level=logical
until docker run --rm --network host postgres:17 pg_isready -h localhost; do sleep 1; done
- if: matrix.build
run: bunx x build --target static --no-preflight
- name: x verify (one part)
run: |
bunx x verify --only '${{ matrix.only }}' \
${{ matrix.shard && format('--shard {0}', matrix.shard) || '' }} \
--json > "$RUNNER_TEMP/part.json" || true
# Outside the checkout: the file is open and empty while the part runs, and `lint` walks the tree.
- uses: actions/upload-artifact@v4
with: { name: 'part-${{ matrix.name }}', path: '${{ runner.temp }}/part.json' }
check: # the one required status check
needs: part
if: always()
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: oven-sh/setup-bun@v2
- run: bun install --frozen-lockfile
- uses: actions/download-artifact@v4
with: { pattern: 'part-*', path: parts }
- run: bunx x verify merge parts/*/part.jsonMake check the single required status. A job that crashed before uploading its part is not a
pass: its steps are missing and merge names them.
Each job sizes its own test width the same way the laptop does (x verify prints it on the step
line, e.g. 3 workers (budget 4.0 GB)): a budget of min(4 GiB, max(2.75 GiB, 25% of the runner's RAM)),
1.25 GiB planned per worker, never more workers than cores. On a 16 GB ubuntu-latest that is 3
workers. ULTIMATE_TEST_MEMORY_BUDGET=6g or ULTIMATE_TEST_MAX_WORKERS=4 override it; --workers
wins over both. The machine-wide slot pool (see Testing)
has nothing to share on a one-job VM; ULTIMATE_TEST_SLOTS=0 turns it off.
x build runs the six static steps before it builds (typecheck, lint, boundaries, filesize,
package-shape, errors) so a bare build never ships an artifact from a tree that does not typecheck.
In a gate that runs those steps anyway, --no-preflight skips them β bun run check does, and so does
the built job above. The default stays safe.