Historical report imported 2026-10-10 from docs/experiments/benchmarks/catalog-discovery-race-findings.md. Original SHA-256: 402a17cce17f75e77eccbc05c4158c5ff9e3cdede4e22d9012a1d069b73efbf4. Measurements and outcomes retain their stated revisions. Links and local environment labels are normalized; $FABRIC_DATA is the mounted experiment drive. See evidence policy.

Catalog discovery-gap counterexample

Status: failure reproduced; unchanged regression passed on candidate repair. Registered protocol (local checkout reference docs/experiments/benchmarks/catalog-discovery-race-protocol.md; not present at origin/main), coordinator receipt (local evidence omitted from stage) and test transcript (local evidence omitted from stage) identify the exact source snapshot, command and protocol hash. No historical passing screen or oracle rule is changed by this finding.

Executed command under root's serial coordinator/resource launcher: cargo test --offline --locked -p fabric-server --lib read_catalog::race_tests::publication_and_reclaim_between_discovery_steps_preserve_coverage -- --exact --test-threads=1 --nocapture. Cargo exit 101, job 7.516980364 seconds; test transcript records one failed, 31 filtered out, 2.95 seconds in target; compilation took 4.13 seconds.

Observed schedule and result

Fixture origin: discovery gap real publication checkpoint reclaim; no retention, seed 0xCA7A10A1. Pre-IO ledger records eight groups/three nodes: 24 Batches and 144 explicit log rows. Thirty unchanged independent-oracle baseline calls exited 0 before the cut; baseline selected custody matched exact producer bytes.

The private unit-test callback ran after catalog Segment enumeration but before tail extend. The actual bounded builder published Segment 1; actual Store and sealer checkpointed/reclaimed journal 1 under unlimited retention. cut.json records after_segments_before_extend, published [1], reclaimed journal 1. The same acquisition then returned Ok with zero Segments, eager Groups, tail entries, blocks or tail paths; oldest_group was 9 and selected Batches 0. No Interrupted was returned. Expected exact custody was 24 Batches.

This is a source-acquisition omission, not demonstrated durable loss: publication retained source bytes, but acquisition combined a pre-publication Segment list with a post-reclaim journal list. Existing movement retry could not engage because no previously enumerated missing path needed opening. The regression assertion allows explicit Interrupted plus exact fresh acquisition; only silent omission failed. The query envelope during the cut was not separately executed, so the result establishes catalog selection failure rather than a measured HTTP answer.

Failure ownership and limits

Preserved root: $FABRIC_DATA/evidence/fabric-work-dabefb93e85f4d04a447b3865ed7d218. Fixture below it: recovery/catalog-discovery-race-01/scratch/storage-361259-0. Keep origin, cut, expected Batch/log-row ledgers, selected custody/source summary, all 30 oracle directories, full Segment state, transcripts and source/protocol snapshots until byte-verified archival. Root owns cleanup; none performed here. Whole-job peak 905,867,264 bytes, sampled scratch 2,151,022 bytes; zero swap. These include compilation/grading/cache and establish no product memory bound.

The schedule uses one journal/Segment transition with empty active state; it is a small deterministic counterexample, not an exhaustive concurrency exploration or minimal single-Batch reduction. The acquisition order was inherited by the catalog extraction. Candidate repair must make the SAME expected-coverage fixture pass on a separately captured revision, preserving this original failed run. Cold-cache/shared-handle variants and other discovery boundaries remain separate admission decisions. No O2/O3 completion or qualification follows yet.

Separate candidate verification

catalog-discovery-race-02 ran the unchanged selector/expected-coverage fixture, exit 0, job 5.513467670 seconds. Its receipt (local evidence omitted from stage) and artifacts (local evidence omitted from stage) record 30 accepted baseline oracle calls, then explicit Interrupted: segment coverage changed during discovery. Fresh acquisition selected Segment 1, oldest group 1 and all 24 expected Batches. The initial failed run remains failed and preserved. Repair compares published label sets around tail discovery and uses existing bounded movement retry rather than returning silent omission.

Whole-job peak 797,671,424 bytes, sampled scratch 1,868,634 bytes, archived evidence 2,158,744 bytes, zero swap; successful scratch removed. This test primes the catalog before the cut. Separate cold/shared controls and the Scan regression subsequently passed their selected schedules; they do not establish full race coverage. Uncached named directories lacking manifests now produce bounded movement errors, a stricter failure behavior; cached metadata still follows the immutable-Segment assumption. No full raw corruption/completeness claim follows. Extra directory discovery cost is unmeasured.