Experiment formal custody prefix red plan 01 - kmosoti/FabricO11y GitHub Wiki

Installation custody archived-prefix counterexample: pre-flight 01

Prospective NOT RUN. C-aud-11 reproduced a leading archived line that the existing sentinel-only conservation check accepts despite absence from all raw Batches and all query rows. X-131 accepts the correction and holds Draft #67.

Clean trust-only head 609368acd7ae3b126b0ba1a82e69281ef1e8acf4 adds exactly one literal regression (47 test lines) and protocol clarification (+41/-3), production unchanged at 409 lines/SHA b100089a531e5cfec9b54a414245ed1e0a4e48c71e765a544a95653eb6adda4d. The same raw CRC, sequence, ACK, projection and inherited-depth controls remain unchanged. Raw server storage equality is not claimed: query equality is projected, raw custody is node-side exact-byte ACK evidence.

The new fixture archive is literal lost-leading-line + pending + unread. It rebuilds archive offsets, cursor consumed-prefix CRCs, raw FAB1/FAC1 and protobuf bytes, and candidate ACK digests. Stopped snapshot reads must admit that raw source before the registered source prefix conservation refusal at conserve. Prediction: one method fails because no ValueError is raised, not because source setup/CRC/ACK fails. No result is claimed until execution.

One serial default-capped source cell only, fresh complete DATA namespace census, 1 GiB reservation under 95 GB admission, 16 GiB memory high/20 GiB max/no swap/1800 seconds. Tiny owned temporary files only, no package, VM, installation, Rust compile, browser or native custody trial. Driver refuses dirty/moved source or another project workload. On failure, preserve raw log/fixture/resource receipts; stop remains mandatory. Root reads #59 after the run and publishes the actual red before correction.

Pins

  • /run/media/kmosoti/data/FabricO11y/results/candidate-build-fresh-8fb6f413179b-01/run-step-custody-prefix-red-01.py SHA-256 ad0ab347212e7b3f2d7c01c3f1556cd92a7ee50017228e06d8d446fc4b552018.
  • /run/media/kmosoti/data/FabricO11y/scratch/installation-custody-witness/tools/qualification/install/custody_witness.py SHA-256 b100089a531e5cfec9b54a414245ed1e0a4e48c71e765a544a95653eb6adda4d.
  • /run/media/kmosoti/data/FabricO11y/scratch/installation-custody-witness/tools/qualification/install/test_custody_witness.py SHA-256 fa4782a9ff25e72d91546207681db8e9f96c4b771472bbbcacb60b41f85c4a61.
  • /run/media/kmosoti/data/FabricO11y/scratch/installation-custody-witness/docs/experiments/formal/installation-release-protocol.md SHA-256 cec645a8f5f029b828526b852385fd411452480a067fbcf0c746499d0d49cd3d.

Exact command

python3 -B /run/media/kmosoti/data/FabricO11y/results/candidate-build-fresh-8fb6f413179b-01/run-step-custody-prefix-red-01.py installation-custody-prefix-red-01 1073741824 env PYTHONPATH=tools/qualification/install:tools/qualification:tools python3 -B -m unittest -v test_custody_witness.CustodyWitnessControls.test_archived_leading_lost_line_cannot_escape_source_conservation

Full trust delta

diff --git a/docs/experiments/formal/installation-release-protocol.md b/docs/experiments/formal/installation-release-protocol.md
index 3109e85..971d726 100644
--- a/docs/experiments/formal/installation-release-protocol.md
+++ b/docs/experiments/formal/installation-release-protocol.md
@@ -176,17 +176,55 @@ establish recoverable delivery; shared-Rust `spool_dump` is a diagnostic.
 `test_custody_witness.py` independently hand-encodes small FAB1/FAC1/protobuf
 fixtures for prospective `custody_witness.stopped_snapshot` and `conserve`.
 The initial named `MechanismAvailability` failure records absence of the
-adapter; it is not a product defect or corrupt-frame counterexample. Actual
-specific refusals remain NOT RUN until that adapter exists. Its unchanged
+adapter; it is not a product defect or corrupt-frame counterexample. The
+[`custody_witness.py`](../../../tools/qualification/install/custody_witness.py)
+adapter now exists; source-control results belong to the linked checker-results
+wiki, while native stopped-custody integration and guest acceptance remain
+pending. Its unchanged
 `CustodyWitnessControls` require complete CRC-checked frames, envelope/wire/u64
 bounds, nonterminal sequence/ACK, full Cursor/Btrfs identity and consumed-prefix
 CRC, last cursor in sequence order rather than maximum offset, complete stopped
 service observations, no-follow owned paths, and exact two-line custody joins.
 Reset source identity, missing/changed pending bytes, premature ACK, duplicated
-or changed query rows and unchanged reboot identity must be refused. ACK events
+or changed projected query rows and unchanged reboot identity must be refused. ACK events
 must match the independently observed candidate node PID/unit/boot/run window
 and exact raw Batch hash; their message alone omits Strand identity.
 
+Query equality compares the projected node label, node ID, sequence, index,
+observed timestamp, string body and string attributes, not every raw LogRecord
+field or nonstring AnyValue arm. Retained source hashes and candidate exact-byte
+ACKs bind the original node-side Batch bytes. This witness does not independently
+reread raw server storage; its raw-byte commitment evidence is the observed
+node's ACK. Host-copy device/inode/mode/uid/gid are drift witnesses, not guest
+ownership-policy admission. Native callers must bind and verify their guest
+observations separately. The raw reader refuses an incomplete active tail,
+including a legitimate torn tail after a crash; it does not repair or tolerate
+that tail as complete custody.
+
+#### Complete archived-prefix conservation amendment, 2026-10-13
+
+Coordination [C-aud-11](https://github.com/kmosoti/FabricO11y/issues/59#issuecomment-6138325736)
+identified a consumed leading line absent from every Batch and query row while
+the two sentinels and candidate ACKs remained valid. Preregister
+`CustodyWitnessControls.test_archived_leading_lost_line_cannot_escape_source_conservation`
+before repair. Its archive is exactly `accept-line-lost-by-reader\n` followed by
+the literal pending and unread lines. Hand-encoded source offsets, Cursor prefix
+length/CRC, FAB1/FAC1 CRCs and exact-byte candidate ACK hashes are rebuilt for that
+archive; the lost line is absent from all raw and projected rows. Snapshot
+admission must succeed, then `conserve` must raise `ValueError` naming
+`source prefix conservation`, rather than rejecting a stale CRC or missing ACK.
+
+For each archived path with a final cursor, the expected old, pending and unread
+projected rows for that path must tile `[0, final_cursor_offset)` exactly in
+source order, with no gap, overlap or duplicate interval. Every interval must
+match the archived newline-terminated body bytes and source device/inode. The
+existing three-row lifecycle baseline and sentinel checks remain required;
+matching them alone cannot establish full archived-prefix conservation.
+Keep every prior control, oracle and gate unchanged. Run the named fixture only
+through the registered serial default1800/full1GiB control cell and preserve its
+actual red before production correction. This amendment changes no product wire
+semantics and records no execution or native acceptance.
+
 Run only through the admitted serial resource launcher: default1800 outer and
 1GiB reservation for synthetic controls. Native cells keep1800 outer,180-second
 lifecycle phases and9GiB whole guest reservation. Preserve failures and raw
diff --git a/tools/qualification/install/test_custody_witness.py b/tools/qualification/install/test_custody_witness.py
index 1f861eb..f2dff4f 100644
--- a/tools/qualification/install/test_custody_witness.py
+++ b/tools/qualification/install/test_custody_witness.py
@@ -166,6 +166,53 @@ class CustodyWitnessControls(unittest.TestCase):
         self.assertEqual(chain[0]['cursors'][SOURCE]['offset'], len(PENDING))
         self.module.conserve(*chain)
 
+    def test_archived_leading_lost_line_cannot_escape_source_conservation(self):
+        # C-aud-11: valid sentinels/ACKs cannot hide an unrepresented consumed line.
+        lost = b'accept-line-lost-by-reader\n'
+        raw = lost + PENDING + UNREAD
+        (self.root / 'allowed.log').write_bytes(raw)
+        def source_cursor(offset):
+            prefix = raw[:min(offset, 64)]
+            return (ld(1, SOURCE.encode()) + vi(2, 7) + vi(3, 9) + vi(4, offset)
+                    + vi(6, len(prefix)) + vi(7, zlib.crc32(prefix)))
+        pending_end = len(lost + PENDING)
+        pending = batch(offset=pending_end, cursor_bytes=source_cursor(pending_end))
+        unread = batch(5, body=UNREAD, offset=len(raw), cursor_bytes=source_cursor(len(raw)))
+        before = self.snapshot('before', [pending])
+        installed = self.snapshot('installed', [pending])
+        drained = self.snapshot('drain', [pending, unread], ack=5)
+        old = [dict(node='spindle-1', node_id=NODE.hex(), sequence=sequence, index=0,
+                    observed_ns=120, body=body, attributes={}) for sequence, body in enumerate(
+                    ('accept-line-before-start', 'accept-line-during-outage',
+                     'accept-line-after-pressure'), 1)]
+        rows = copy.deepcopy(old)
+        for sequence, body, start in ((4, PENDING, len(lost)), (5, UNREAD, pending_end)):
+            rows.append(dict(node='spindle-1', node_id=NODE.hex(), sequence=sequence,
+                index=0, observed_ns=123, body=body[:-1].decode(), attributes={
+                    'log.file.path': SOURCE, 'log.file.device': '7', 'log.file.inode': '9',
+                    'log.file.offset.start': str(start), 'log.file.offset.end': str(start + len(body))}))
+        queries = dict(before=old, after_drain=copy.deepcopy(rows),
+            after_reboot=copy.deepcopy(rows), boot_before='literal-boot-a',
+            boot_after='literal-boot-b', pending_body=PENDING[:-1].decode(),
+            unread_body=UNREAD[:-1].decode())
+        events = [dict(unit=PROCESS['unit'], pid=55, boot_id=PROCESS['boot_id'], at_ns=150,
+            message=f'delivery sequence={sequence} sha256={hashlib.sha256(payload).hexdigest()} '
+                    f'status=ack committed_through={sequence} elapsed_us=1')
+            for sequence, payload in ((4, pending), (5, unread))]
+        self.assertEqual(before['logs'][SOURCE]['bytes'][:len(lost)], lost)
+        self.assertEqual(before['cursors'][SOURCE]['offset'], pending_end)
+        self.assertEqual(drained['cursors'][SOURCE]['offset'], len(raw))
+        for snapshot in (before, installed, drained):
+            self.assertEqual(snapshot['cursors'][SOURCE]['prefix_crc'],
+                zlib.crc32(raw[:snapshot['cursors'][SOURCE]['prefix_len']]))
+            self.assertFalse(any(row['body'] == lost[:-1].decode()
+                for source in snapshot['sources'] for row in source['rows']))
+        self.assertFalse(any(row['body'] == lost[:-1].decode()
+            for observed in (queries['before'], queries['after_drain'], queries['after_reboot'])
+            for row in observed))
+        with self.assertRaisesRegex(ValueError, 'source prefix conservation'):
+            self.module.conserve(before, installed, drained, queries, events)
+
     def test_complete_frame_and_envelope_refusals(self):
         wrong_wire = ld(1, b'\x01') + batch()[2:]
         variants = [batch(version=2), wrong_wire, batch(node=NODE[:-1]),