Experiment formal release service query run 03 - kmosoti/FabricO11y GitHub Wiki
Exact-candidate service and fixed-query run 03
Recorded 2026-10-12 for #42, in epic #33, which supersedes historical #32.
Six registered main cells and four registered full fixed-query cells exited 0. All 13 main gates and all eight fixed-query gates passed in every corresponding cell. These results are historical harness shakedown evidence on the exact superseded packages below and do not count toward the new candidate’s B2–B5 gates, as required by the owner decision. Remote forwarding, 30-minute outages, fault controls, the successor two-hour soak, packaged browser/role acceptance and the fresh-owner walkthrough remain open; no deployment profile or release is qualified.
Identity and protocol
- Artifact source:
a5dbda7a4fd6465c94f6505b3214cbcbc2130c82; minimized compiled source:f23830d5f70dd3029d540def41fdaf50fe0facf3. - Debian
0.1.0~alpha.1SHA-256:54cddf548c1a307aade8c3ac4332ede2007e2bc9603ab0cba72d8297287d6cee. - RPM
0.1.0-0.alpha.1SHA-256:982ad6923e17f64b35700dcfafb2994ed68e32b2c6b368589a8b6a0d550cac79. - Source manifest SHA-256:
b9944726e191e1bec0fe6abdcf3fbbbd905504c90a076167c09c33860e6f09e2; bundle SHA-256:abd1ea6458b16931af226f84b29ec226298a9084c4b5a4062e9166a386d1d9fc. - Main harness:
8484493696a3e3e743889a83f14a60df54305aa9; full fixed-query harness:8cfc06e8efe2decffa21b7dbee854b69df344e6b. These separate harness revisions do not change the compiled candidate. - Procedure: registered release service/query R3. Clearing-v2 was registered and prospectively selected before the six main measurements. Page pacing and installation-wide reader corpus amendments were registered before the affected successor cells.
- Full query plan: prospective command plan. The independent delivery/query oracles, quotas, fixture coordinates, latency thresholds and production binaries were unchanged.
Commands and environment
Each native command ran serially on the owner Fedora workstation through the resource launcher. The private command receipts record the complete arguments, recorded CPU topology/kernel and resource controls. Server and companion used CPUs 4–5; worker/browser/supervisor used CPUs 2–3. A real native edge, dedicated server companion and active authenticated console were present in each main cell.
python3 -B tools/resource_group.py --delegate -- python3 -B tools/qualification/release_main.py \
--deb-receipt "$DEBA/receipt.json" --rpm-receipt "$RPMA/receipt.json" \
--server-cpus 4-5 --worker-cpus 2-3 --tier {10,100} --seed {0xA11FA001,0xA11FA002,0xA11FA003} \
--backlog-rule clearing-v2 --out "$R/service-opening-r3-main-tT-sS-01"
python3 -B tools/resource_group.py --delegate -- python3 -B tools/qualification/release_query.py \
--deb-receipt "$DEBA/receipt.json" --rpm-receipt "$RPMA/receipt.json" \
--server-cpus 4-5 --worker-cpus 2-3 --mode {segment,journal} --seed 0xA11FA00S \
--out "$R/service-opening-r3-query-MODE-sS-02"
The braces above compact the matrix; the ledger retains each actually executed command and exit. $R is /run/media/kmosoti/data/FabricO11y/results; $DEBA is its installation-package-build/service-opening-deb-a-retry subdirectory and $RPMA is packages/service-opening-rpm-a. Every Segment seed ran; journal mode used seed 1.
Six main cells
Each cell had 15 seconds warmup and 120 seconds measured. ACK latency uses creation-to-ACK for simulated sources and same-boot source-acceptance-to-ACK for the native edge/companion. The combined column merges those measured cohorts only within the same cell. First-send-to-ACK and native Spool commit timing are retained separately; seeds are never pooled.
| Cell | Sim / edge / companion ACK p99 (s) | Combined ACK p99 (s) | Highest visibility p99 (s) | Longest config apply (s) | Server / edge peak RSS (MiB) | Exit |
|---|---|---|---|---|---|---|
| r3-main-t10-s1-01 | 0.011353 / 0.021933 / 0.010163 | 0.021709 | 1.005758 | 4.679302 | 27.746 / 5.512 | 0 |
| r3-main-t10-s2-01 | 0.018182 / 0.022655 / 0.013029 | 0.021780 | 1.012683 | 4.724563 | 27.648 / 5.609 | 0 |
| r3-main-t10-s3-01 | 0.010857 / 0.022080 / 0.013652 | 0.021775 | 1.007985 | 4.701439 | 27.559 / 5.547 | 0 |
| r3-main-t100-s1-01 | 0.094782 / 0.029165 / 0.009181 | 0.094446 | 1.014836 | 9.413822 | 92.461 / 5.656 | 0 |
| r3-main-t100-s2-01 | 0.094127 / 0.026065 / 0.010131 | 0.094014 | 1.012239 | 9.297091 | 91.191 / 5.668 | 0 |
| r3-main-t100-s3-01 | 0.094791 / 0.030839 / 0.008167 | 0.094727 | 1.010032 | 9.372630 | 91.359 / 5.691 | 0 |
All original ACKed source bytes were retained and independently matched to recovery; sampled queries, native logs and native traces were exact. The private ledger retains generated/ACKed Batch counts, independent offers, per-population sample sizes, timestamp mapping, every owner command and its requested/response times, individual configuration-apply latency and active-console body completions. Owner commands were serialized, so this does not claim simultaneous fleet configuration updates.
Backlog limitation: the historical sampled-v1 edge diagnostic failed in five of six main cells. Its failures and continuous first/last mean observations remain visible. The admitted clearing-v2 rule passed (bounded busy periods and an empty queue in every five-second measurement window), with no unACKed final records. This is not a claim of stationary backlog or nonincreasing sampled means.
Four full query cells and trace fixture
Each full fixture contained 100 identities and 10,000 original Batches: 500,000 logs, 500,000 metric points, plus 1,000 distinct trace IDs and 3,000 spans. Every cell graded 200 logical queries: 20 repetitions of five query kinds before restart and 20 of each after restart. Each query includes every continuation page and pacing wait in its elapsed time. A refused request is not retried or excluded.
| Cell | Exact answers | Highest p99 before / after restart (s) | Server peak RSS (MiB) | Independently decoded trace IDs / spans | Exit |
|---|---|---|---|---|---|
| r3-query-segment-s1-02 | 200/200 | 1.220328 / 1.219419 | 79.676 | 1000 / 3000 | 0 |
| r3-query-segment-s2-02 | 200/200 | 1.208010 / 1.206994 | 79.773 | 1000 / 3000 | 0 |
| r3-query-segment-s3-02 | 200/200 | 1.223049 / 1.218567 | 79.648 | 1000 / 3000 | 0 |
| r3-query-journal-s1-02 | 200/200 | 1.179402 / 1.180256 | 333.113 | 1000 / 3000 | 0 |
| Cell / phase | source logs p99 (s) | rare text p99 (s) | source counter p99 (s) | fleet counter p99 (s) | trace p99 (s) |
|---|---|---|---|---|---|
| r3-query-segment-s1-02 / before_restart | 0.222666 | 0.224149 | 1.220328 | 0.168351 | 0.150005 |
| r3-query-segment-s1-02 / after_restart | 0.218040 | 0.222102 | 1.219419 | 0.167794 | 0.150820 |
| r3-query-segment-s2-02 / before_restart | 0.233992 | 0.219771 | 1.208010 | 0.189636 | 0.150455 |
| r3-query-segment-s2-02 / after_restart | 0.232124 | 0.221307 | 1.206994 | 0.179077 | 0.149369 |
| r3-query-segment-s3-02 / before_restart | 0.225437 | 0.217657 | 1.223049 | 0.179491 | 0.162331 |
| r3-query-segment-s3-02 / after_restart | 0.226365 | 0.227230 | 1.218567 | 0.179845 | 0.149687 |
| r3-query-journal-s1-02 / before_restart | 0.170522 | 0.168803 | 1.179402 | 0.250548 | 0.153798 |
| r3-query-journal-s1-02 / after_restart | 0.170567 | 0.156900 | 1.180256 | 0.233657 | 0.163372 |
The trace fixture is separately decoded from original independent producer bytes. Every trace has one root and two children with exact parentage and distinct span IDs. Custody checks cover all 1,000 trace IDs/3,000 spans; the query oracle separately grades the registered 20 trace-ID repetitions per phase. This does not claim an individual query of all 1,000 IDs.
The all-telemetry reader corpus includes the server companion, independently joined from its retained Spool bytes and exact recovery metadata. The first-page native acceptance/ACK bounds admit only independent source prefixes; the query response never supplies expected telemetry. Unchanged row, freshness, retained-window and continuation checks passed. Both authenticated UI phases completed with the recorded body-completion witness.
Preserved failures and controls
- The original main enrollment smoke remains failed HTTP 429. Its command-admission repair and independent red control are in command admission run 01.
- Full Segment seed 1
-01remains failed HTTP 429 after 42 completed logical queries/66 pages. The paced smoke-02remains failed because its obsolete fixture-only authority omitted installation-wide companion metadata. Neither is relabeled as a pass. - Fixed-query caller run 01 retains those failures, the independent unpaced and corpus-omission controls, 59 passing targeted checks and the fresh smoke
-03. Smoke is admission evidence only. - The closeout validator was challenged with a forged command-receipt digest. Its actual exit 1 reported
closeout cell receipt binding mismatch; the positive validation then exited 0. Both outer groups stopped. This check tests receipt binding; it is not another native workload.
Containment, storage and evidence
The native fixtures tightened the launcher’s aggregate envelope to 6 GiB with separate server/companion, edge, browser and supervisor groups, zero swap, finite CPU/task limits and the default 1,800-second deadline. Every native admission reserved 5 GiB on the mounted data drive. Namespace-aware disk census never skipped subordinate-UID container files; the 95 GB stop and 100 GB aggregate ceiling were enforced. All ten invocations confirmed immutable inputs/harness, stopped descendants and removed their owned success scratch. Raw failure evidence and its explicitly retained scratch remain private.
Maximum aggregate namespace-aware file-byte census across these ten cells: 90,593,601,655 bytes. Owned compiler-library cleanup between query cells removed 1,032,026,053 bytes under separate capped, inode/hash-guarded receipts; published packages and raw evidence were preserved.
Private ledger: results/service-opening-candidate/service-query-completion.json, SHA-256 1956f11941fb53be1c25b95aefcbc8de2c55cffc7b76fad243d473aaae8a6b52. Validator SHA-256: c39ec9ad4e20167978eff4b686d80cf53abaa073bc2e1c02a044403eaa51657b. The ledger records all commands, exits, host/resource observations, file inventories, harness hashes, per-cell metadata and cleanup. Raw evidence is stored under the mounted data root; no credentials, private telemetry, binaries or source fixtures are published here.
| Cell | Summary SHA-256 | Runtime SHA-256 | Command receipt SHA-256 |
|---|---|---|---|
| r3-main-t10-s1-01 | 3cb43fd3cbffb0cb67eaf6fee9cac207f886ce73dadbb28cc1cb462dffe3b722 |
58a210e73ee69484d4898016dca07ca89e759de2d01756c24b3cf647c4ef41a6 |
46f13fc669451e3fb3abcea882a3101943255ec3ac99212dbc0b07a50bbeadc0 |
| r3-main-t10-s2-01 | c78f3ccaf16087215452c1a90d8ea6e1cb4aa0445dee51d34894d08f67301604 |
d529445d42d92f3cb6f534be99170133c10431fd19a55df7cfa7acda76a698c4 |
99ad75ccbe3dadb9198111faf1a55344de03c9dd05ae4f4978c9573c5ebcfddf |
| r3-main-t10-s3-01 | f790af659f20fc8d78df3aedc84e7990f26f9d693e8188ddf56441fbb7b8273b |
34a1ab4e02839c56ef77b20e19524a8a339e84692c9ec4bd0a5253d1b94943a1 |
f9908a8de76b67a7f707371176cc2b3118ae3457ab3cc90ebd31667e57858daf |
| r3-main-t100-s1-01 | 26764dad2a9cce256d260c5bf671743c0fc7a5e392f0d71fb5155eb9831ec936 |
86f8cfd4b3cbd31648c782f943218dd4d528d4879b24773a928db2f99acba2f2 |
2ce7ecbb3f4a486f8fc571020705d4ba0a6ab0c9c633a71919e7435e98e35e35 |
| r3-main-t100-s2-01 | 964ae2fc36c84ce86b6c3393f7262230bc44eb744f693c4360e0dd6d016050f1 |
dabddc4f4b97c094645775bab635c8848614782c493c030b904ab6953979e440 |
c8453daee60a05f1a6b0dd79fce4f329c621a07ef8125a4f526a6f5ca28fd7f6 |
| r3-main-t100-s3-01 | e3f8039ab39b4c73bacc04cc9042d67afeaff54009e3761a8076c619665f55a9 |
389fb8dd379914367a402ad29715e9961306d70c2a537fe30b14a5113a4748c7 |
2f7e24cd2f7ddef4be11a98cee2937061107655c172d98562f218438580f5655 |
| r3-query-segment-s1-02 | 85bdba4296a6a89f7eda18e43a804e454bc14e51c9b7c91ce70e241a393ec793 |
cb51a88e5612778d36c1bebe69fec0e03ee13c9ddad3b1d84dd395a6e09d6d46 |
6f8b0c88d208615bb5f705fd36f839e136a3ded419676b6c2f63da7af2084700 |
| r3-query-segment-s2-02 | 2bd03ce8635c2254d1bbf5ba58bd4937b523dc2e602c4c1b166d9c7317117177 |
06bb67066e01295879c41bfac98c9278762fa90924379d78f6752949e07eb837 |
edb899bbc6bdb7782ffadcd3191b1bfd0eed90cac9cb9c2dabc38556b7494468 |
| r3-query-segment-s3-02 | a2247c235153ffbd18cd70dcca75fc60919c385c23555543b60ad040217a956d |
f232960ea3cd7dcb853d40e4304e15979b7868c0adfa0c90db4e3a28aac4c350 |
9fff8133d709233edb3ae4534f1cbb679056c8195aafc494834fcd6766d7a5c9 |
| r3-query-journal-s1-02 | 4edfc881d4e448f93db1be3495d373e6c646a01496fd1646b7d5a043a87118f8 |
f5c2360dc7799dc311ee6ca8eb552a3b2a7c063d3c446732191042e8987eba73 |
c7d4abdc51aed784950321328d321f61305fc724f88054d77fe68350fba0ce12 |
Decision
Keep #42 open for the new candidate. After #57 and #58 merge, independently rebuild two console/package pairs and rerun #41/#42; carry none of these superseded package results into #47. Local pinned Debian/Fedora VMs and longer outage/soak deadlines are owner-approved. Remaining prerequisites and evidence gates still apply; no release tag is produced.