Experiment benchmarks catalog algorithm round findings - kmosoti/FabricO11y GitHub Wiki

Historical report imported 2026-10-10 from docs/experiments/benchmarks/catalog-algorithm-round-findings.md. Original SHA-256: 5e5f0c69af3975e459a44a6e84ad3a6e86f749c78be2a397f6e039610e0e852f. Measurements and outcomes retain their stated revisions. Links and local environment labels are normalized; $FABRIC_DATA is the mounted experiment drive. See evidence policy.

Commit planning, query selection and collection ownership

The three labs produced smaller representations of work Fabric already does: remember distinct bindings instead of every accepted offer, order query keys without a second payload tree, and transfer Strings instead of copying them. The retained implementation follows the registered round (local checkout reference docs/experiments/benchmarks/catalog-algorithm-round-protocol.md; not present at origin/main) and small-group follow-up (local checkout reference docs/experiments/benchmarks/catalog-small-binding-protocol.md; not present at origin/main). No wire, sync, query-order or source-cursor contract changed.

Results and the rejected candidate

Ratios below are candidate wall time divided by the frozen original's time; lower is better. Each is the median of three paired ratios. See the first summary (local evidence omitted from stage) and compact-prefix summary (local evidence omitted from stage) for every arm and every guard, including unfavorable observations.

Decision kernel, distinct sources Immediate secondary indexes Retained compact prefix then indexes Time reduction from original
1 0.880 0.741 25.9%
8 1.436 0.885 11.5%
32 1.138 0.755 24.5%
256 0.418 0.427 57.3%
2048 0.084 0.088 91.2%

The first candidate failed its small-workload guard. The follow-up retains the first binding inline and at most 31 secondary bindings in a compact list. It deduplicates repeated accepted bindings and consumes that list into two maps when a 33rd distinct Spindle appears. All original guards and the added 32-source guard passed. Single-binding cases also passed. The large same-Strand cases are stress fixtures; normal one-request-in-flight delivery does not produce that group population. The probe supplies empty durable facts, so already-bound steady-state service cost is not measured. Production code preserves lazy forward lookup when an explicit durable credential binding is available.

The kernel changes worst-case repeated linear binding work to a bounded prefix followed by indexed lookup. The fixed 32 crossover is a local hypothesis supported by this screen; neither a global optimum nor a prediction of service throughput is established. Journal append/sync and transport are outside these timers.

Selection fixture Dense-slot/original median ratio Time reduction
Descending, 16-byte bodies, 21 slots 0.719 28.1%
Descending, 16-byte bodies, 1001 slots 0.637 36.3%
Descending, 1024-byte bodies, 21 slots 0.670 33.0%
Descending, 1024-byte bodies, 1001 slots 0.618 38.2%

All 96 selection arms preserved exact rows/order. The heap carries key, admission ordinal and slot; payload slots are reused on eviction, removing the payload BTreeMap. Empty selectors allocate no heap/row storage. The logical row count is bounded by capacity, while vector allocation may round upward. Ascending 21-slot medians were 7.4–8.2% slower; some individual equal-time/mixed pairs were also worse. All registered median guards passed. Replacement-heavy results must not be applied to every query, full storage decoding or service latency.

Moving log bodies and paths into OTLP produced byte-identical output. Median encoding ratios were 0.879 for 1024 lines with 128-byte bodies and 0.949 for 192 lines with 4000-byte bodies. The large-body pairs were 1.078, 0.689 and 0.949: the narrow median criterion passed, but variability limits a broad speed claim. Inputs are constructed outside timing; consuming/dropping owned inputs is inside candidate timing while borrowed inputs remain alive outside the legacy timer. Pointer controls independently demonstrate eliminated String copies. Log-only and failed-host-sample paths now retain history via None instead of cloning it.

Correctness evidence and implementation

The first round ran 10 core delivery tests, six collection ownership tests and three selection tests; all returned exit 0. The refined core ran 12 delivery tests, all exit 0, giving 21 targeted controls on the retained mechanisms. The ignored timing tests ran separately in release mode. The evidence includes:

  • Every delivery outcome, seed 42 with 64 histories of 512 offers, changing durable facts, first-forward/every-reverse binding preservation, 32/33 promotion, and repeated accepts that must not grow binding state.
  • Frozen legacy selection and independent stable full-sort comparisons, checked through every prefix of 4096 rows for five orders and capacities 1/21/1001. Reversed ties and altered thresholds are rejected. Rejected, evicted, returned and cancelled payloads are dropped once.
  • Exact legacy protobuf bytes plus explicit Unicode/numeric fields, live pointer identity and a borrowed-clone negative control, unchanged history on quiet, log-only and failed-host-sample paths, and cursor/history ownership across successful and refused Spool commits.

Production code is in core delivery, query selection (local checkout reference crates/fabric-server/src/query/selection.rs; not present at origin/main) and native collection. Supporting architecture: delivery, retained history, and Spindle.

Commands, resources and cleanup

Both preregistered launch commands executed through the resource launcher and serialized coordinator, under 20 GiB memory max, 16 GiB high, no swap and the 30-minute outer deadline. Both returned exit 0. Compiler/tests/benchmarks and their validators shared the containing cgroup. Exact commands, exit codes, post-format source hashes and snapshots are in the round receipt (local evidence omitted from stage) and follow-up receipt (local evidence omitted from stage).

Job Elapsed Cgroup CPU Cgroup memory peak Sampled scratch peak
Round 01 80.093 s 195.650 s 1,918,869,504 B 702,380 B
Compact prefix 01 13.513 s 27.099 s 367,767,552 B 0 B

CPU and memory include compilation, tests, filesystem cache and all driver work; they are not per-arm application CPU/RSS. A zero scratch sample does not mean no file was ever created. The coordinator round (local evidence omitted from stage) and follow-up (local evidence omitted from stage) receipts record raw cgroup I/O, pressure, limits and memory events. Both had zero OOM/high/max events, zero swap, no surviving descendants and successful owned scratch removal. Build caches and owned scratch use $FABRIC_DATA; no remote workload ran.

The follow-up's final admission measured 1,815,064,576 aggregate allocated bytes and 1,069,060,096 conservative query-evidence bytes before receipt closeout writes, within the existing caps. It did not reset the frontier; about 2876.12 seconds remained before final verification. Raw evidence and failures are retained; successful temporary fixtures were removed.

Final fast and manual documentation gate commands, source hashes and exact outcomes are recorded in the verification receipt (local evidence omitted from stage). The coordinator receipt (local evidence omitted from stage) records verification resource use and cleanup. The Bun copy is removed only after exact comparison against the existing retained archive member and a saved reference. No deployment qualification is claimed by these finite runs.

Remaining useful angles

These changes share a principle: separate rank/binding facts from payload owners, and change representation only when population justifies its overhead. Further work should measure the production workload before selecting another constant.

The oversized-line skip-only progress question remains an unrun hypothesis: a long continuation can return no payload, discarding prospective cursor progress until a later metric/payload commit. Resolving it requires an explicit valid-Batch choice. Chunked newline scanning, avoiding String-per-byte hex formatting, and consuming decoded OTLP rows instead of copying them remain separate candidates. End-to-end ACK, acceptance-rate, query-latency and RSS improvements from this round have not been measured; the earlier deferred service matrix remains deferred.