Experiment formal native storage census results 01 - kmosoti/FabricO11y GitHub Wiki

Native storage census: finite correctness and metadata experiment

The pinned native census matched the original Python reference and an independently enumerated classification manifest. Six focused source tests, the representative retry-counter mutant, and a registered seven-pair metadata experiment provide the evidence below. No owner walkthrough, alpha acceptance or whole-drive qualification ran; caller adoption remains separate.

Source, contract and artifact

All project commands used the retained Rust 1.99.0 toolchain and resource launcher, default 16/20 GiB high/max, zero swap and 1800-second deadline. The entire DATA tree was counted before each admitted cell against 95 GB plus its full reservation. Source-focused/private-target build cells reserved 3 GiB; the finite comparison reserved 1 GiB. No simultaneous local project workload ran during measurement. Shared dependency downloads remained cached, while Cargo targets were isolated by source and variant after the cache counterexample below.

Actual correctness observations

The classification fixture covers ordinary and empty files, a hardlink counted per pathname, a 65537-byte sparse file counted by logical length, nested payload, file and dangling symlinks, directory and outside-directory symlinks, and a FIFO. Native and Python each exited 0 and returned the exact same six fields: 65596 total data bytes, 65564 regular-path bytes, 8 file paths, UID 1000, GID 1000, attempt 1. Both matched the manifest. Outside symlink targets were not descended.

The six source tests actually passed at 9e3b6e0, and again after a clean isolated-target build at measured 10540df: atomic checked counters; exact owned/runner-control retry classification; real exact-root removal; file/link/sparse/hardlink/FIFO accounting; unknown/sibling/permission/I/O/exhaustion refusal; and real second-file unlink with complete partial-counter reset. Focused Clippy and formatting checks exited 0. These are source controls, not evidence that an entire live DATA scan or owner acceptance run occurred. File unlink and root removal use real filesystem changes. Unknown/sibling paths, permission/I/O errors and repeated transient exhaustion use injected error cases; they are not real disk/ACL failure trials.

A separate deliberate counter mutant moves counter initialization outside the retry loop and passes accumulated totals by reference. Its actual cargo test --locked -p xtask census exited 101 at 03:51:49.493496 UTC: only real_second_file_unlink_restarts_and_discards_partial_totals failed, with totals 36/36/9 versus 32/32/8; the other five tests passed. The failed fixture and outer evidence were retained. This is a named defect-detection observation; the failed command remains recorded as 101. It is not credited from a compiler/setup error.

Registered warm metadata corpus

The separate performance corpus contains 20000 regular files of 64 bytes across 10 directories: 1280000 data/regular-path bytes and 20000 paths. One warm-up per implementation preceded seven pairs alternating execution order. All 16 runs exited 0 and matched every expected field.

Metric Native Original Python
Median harness wall time including GNU time wrapper 31.530237 ms 74.323464 ms
Median GNU time maximum RSS 2764 KiB 11628 KiB

The median of seven paired wall-time reductions is 57.858%; reduction computed from the two medians is 57.577%. The exact seven raw pairs and process-level GNU time records are retained. CPU user/system values have only 0.01-second resolution and do not support precise CPU speedup claims.

The measured boundary wraps each census process with GNU time, including process/interpreter start and the wrapper; fixture creation and result parsing are excluded. Host: Intel Core i7-10750H, six cores/twelve logical CPUs; the owned fixture was on the mounted ext4 DATA drive. These warm fixed-corpus results do not establish cold-cache, full-drive or differently shaped filesystem performance. No optional backend/syscall variant search followed the registered result.

The comparison exited 0 with whole-cgroup stop at 04:06:45.091537 UTC, source/helper/ELFs unchanged. Owned fixtures were removed only after their immutable report was retained. Candidate report SHA256 557649874b5dd6484dbf961393a381017f23093495211dae9d0bfc2200525ad2; wrapper SHA256 409a551c52df28187bff46996b8e2952ed2ea0c1ba8de93258d19cd773fdd674.

Preserved failures and cache/source join

Earlier baseline Clippy rejected a complex return tuple and an unnecessary borrow; the minimal alias/borrow correction was checked. Initial candidate compilation failed because pinned libc did not expose RAMFS_MAGIC; the corrected explicit constant is bound to /usr/include/linux/magic.h, without substituting CRAMFS or removing the disk-backed-root guard. A comparison-driver compile then rejected an incorrectly dereferenced slice item; the literal comparison was fixed. Each failed run remains in its original namespace, with later cells not run until correction.

A subsequent candidate test unexpectedly repeated the mutant's 36/36/9 failure. Cargo reported a 0.06-second test build with no compilation, then executed a shared-target test ELF whose embedded absolute root was scratch/storage-census-mutant, although the production census source was byte-identical to the earlier six-test passing source. The actual unexpected 101 and binary were archived: ELF SHA256 de251eded7ac465a50eb63b56d0371a8e886abf01c9bb81b73e3ff2534bdd9c8. It is classified as a source-to-cache-artifact join failure, not a supported production regression or an ordinary pass.

The prospective 10540df amendment requires source/variant-specific DATA Cargo targets for these worktrees. Because the launcher sets an outer target directory, the correction is an explicit inner env CARGO_TARGET_DIR=ABS_SOURCE_VARIANT_DATA_TARGET argument. No launcher guard or production census behavior was changed to resolve the cache failure. The isolated retry actually recompiled the correct source and passed six tests, then built and pinned the release artifacts used above. The original mutant result remains valid: its actual failed ELF was from the mutant worktree. Earlier frozen product packages are not rebuilt or invalidated by this unrelated tooling counterexample.

Reproducible records and remaining scope

DATA records reside in results/storage-census-baseline-01, results/storage-census-candidate-controls-01, results/storage-census-mutant-controls-01, and results/storage-census-comparison-01. They include literal environment-prefixed commands, source/driver/tool hashes, complete logs, reservations and resource/whole-stop receipts, preserved format/compile/cache failures, failed fixtures, isolated target paths, artifact pins and actual raw outcomes. Classification/measurement reports and raw stdout/stderr/GNU time/hardware/filesystem facts reside in results/native-storage-census-candidate-01; actual-summary-01.json independently recomputes the reported statistics.

The concrete comparison pre-flight binds the actual two ELFs and fresh output. Subsequent Rust caller wiring still requires its own exact owned-path bindings, independent controls and source/main gates. At the comparison boundary, alpha acceptance was owner-paused. The broader 200-program migration inventory remains incomplete; the later continuation boundary is recorded below. Nothing here relabels prior interrupted acceptance as passed.

Owner binding and adapter integration

The native owner-binding source at 56b37cfec5dae486a7fd4c6e30f15b6d71f53ebb completed X300's seven source commands with exit 0 and whole-cgroup stop at 2026-10-18 04:21:45.054743 UTC: 73 Rust tests, all-target/all-feature Clippy, formatting, frozen documentation dependencies, links, 55 documentation probes and 10 hooks. The source was clean before and after. Wrapper SHA256 ef3e74f84700802d6a4da821018433b1e02ba286e95ba7b04a625b6d5a47161d. Earlier formatting and Clippy failures remain preserved; these results belong to source 56b37cf, not subsequent integration revisions.

A contained release build at that source exited 0 and stopped at 04:24:49.095294 UTC. Its immutable mode-0500 CLI is results/storage-census-owner-binding-01/artifacts-01/xtask-owner-binding, SHA256 c78253ec88056c1aa947ece85bb8c8810652a1e7ec3356ee360244be3ba6ba13; build wrapper SHA256 5dd0f288731782cafa5fc31be83423a7c8c26dbfe6a1d0aea7f6c29ba8c26e70. No performance measurement is attributed to this newer binding revision.

The thin adapter preserves the original owner's positional source-owner argument, runtime map and all other driver AST nodes. It adds independent metadata loading and replaces only the old census subprocess with the pinned native CLI in the original podman user namespace. Frozen adapter SHA256 e155b2dd28d0929c340138be13ec7544139090439150456e0dc582baeabb9bf3; original driver SHA256 c9fe723ec3591827700e67cc2cef74c5118d28e8dc8ca0ec49e3246ede8e8aa4 remains unchanged.

X304's result records eight expected controls, outer exit 0 and whole-cgroup stop at 04:36:02.978864 UTC:

  • Both literal original plan command vectors returned the complete six-field DATA census with UID/GID 0 and attempt 1. Only the unresolved admission-hash placeholder was replaced with a clearly labeled pure 64-hex fixture.
  • Wrong label, duplicate run ID, unresolved raw-command hash and another program each produced actual native exit 2, the named parser diagnostic, and no success output.
  • A null independently declared admission hash and a mismatched independently declared label each raised the adapter guard exception before any namespace/native subprocess invocation.

The composite control executed the actual new metadata initializer, unchanged require() and actual replacement census() extracted from the frozen adapter AST. All remaining original AST nodes were compared for equality but not executed. This does not establish execution of the original runtime-map guards, admission-content verifier, delegated owner command or any VM. The actual source-owner admission templates remain null and non-runnable until the original #45 and final admission requirements are satisfied; the pure fixture is not admission authority.

Terminal record results/owner-census-rust-adapter-01/terminal-01.json, SHA256 547e54e8440ab0cb3ef64c8c50d2d9a051e053bf831b59013d0e8eb2cbb7ad6b, binds the raw per-case arguments/stdout/stderr/exits, preparation, adapter, plan and resource receipt. Resource SHA256 a2efd0f9008669149ba78291a0265c08e62bb4014b4da0149a122ea363f33e7a; no swap or surviving workload is claimed beyond the actual stopped receipt.

These finite verification prerequisites satisfied the owner's conditional continuation instruction. Alpha acceptance resumed at the fresh #42 continuation-preparation boundary, preserving its eight completed full cells and interrupted segment3-01. No #46 owner VM or full-soak result is implied. The later source integration at 29ef1861a7364f5271689220e37be9049b3593d6 includes the consolidated verification source and documentation; its own final source gates remain pending. The frozen 56b37cf CLI and original attribution above are unchanged.