Stack: ork-driver feat/ork-async-stream, src/npu.c + examples/test_f16colsplit.c. Board RK3588 (10.3.0.236), kernel 6.1.115-vendor-rk35xx, DDR 2112 MHz / A76 2.4 GHz, governors=performance, single-stream. Kernel source cross-referenced: rockchip-linux/kernel develop-6.1 drivers/rknpu/{rknpu_job.c, rknpu_iommu.c, rknpu_drv.c}.

The fp16 doorbell colsplit (ork_dyn_begin_colsplit, dt==DT_F16, N-columns split across 3 cores, K-sliced with host f32 accumulate) intermittently dropped a K-slice — at K=3584 essentially every op. The happy path is 2.6× vs single-core and bit-exact, but the drop made it unshippable, so it was gated off (ORK_F16_COLSPLIT, default off). The int8/int4 colsplit recover their drops cleanly (mc_recover_resubmit); fp16 did not, and recovery attempts kept hard-wedging / kernel-panicking the board (≈5 power-cycles this session).

Conflating these is what cost the most time. The recovery protocol is correct for the benign class; it can't help the wild-DMA class because there's no damage to undo — the wild write already landed.

The 3 cores each loop their K-slices independently. The NPU prefetches the next slice's weight base (Bb[ks+1]) while Bb[ks]'s fp16 weight-DMA is still draining → two distinct dma-buf fds on the CDMA at the same instant → the mis-addressed fetch ("cdma wild"). fp16's 2-byte weights double the fetch bytes vs int8, which tolerates the identical pattern.

Decisive evidence: every traced drop was on the boundary slice — Kslice=1 at Sk=2, never Kslice=0 (campaign4/5/7, live-traced via ORK_F16_TRACE). That is the textbook cross-boundary-prefetch signature. Same-buffer concurrency (3 cores reading different column regions of ONE buffer) is benign — only cross-buffer (distinct fds) wilds. A CPU lockstep barrier can't fix it: it gates the software loop, not the hardware prefetch, which latches the next base before the barrier releases.

The kernel mechanism (source-confirmed): any BLOCKING submit that can't dispatch into a post-drop NPU (e.g. the nc=1 fallback) enters the kernel's 60→120→180 s continue_wait, then rknpu_job_abort runs:

rknpu_iommu_domain_put(rknpu_dev);   // release the device-GLOBAL domain ref
msleep(100);                          // ... 100 ms with the IOMMU torn down but DMA not halted
rknpu_soft_reset(rknpu_dev);          // only NOW halt the engine

The device-global iommu_domain_refcount gates domain switching (rknpu_iommu_domain_get_and_switch: switch/iommu_detach_device+attach only if refcount==0). A dropped job's double-cleanup (timeout_clean's cleanup_work racing the late completion / abort) over-decrements it → a later submit reads refcount==0 while a job is still live → detaches the IOMMU under a live matmul → wild write into kernel RAM → slab corruption → Oops/panic in innocent processes (percpu_ref_put, rb_insert_color, shmem eviction). rknpu_job_timeout_clean is safe only because it soft-resets before teardown; abort reverses that order. Fast-detect + reset-first did not prevent it (the wild write precedes detection; the abort race is separate).

Concatenate all K-slices into one buffer w->Bbc (built lazily, Sn==1 only). There is then no dma-buf boundary — the "next slice base" is just the next address in the same fd/mapping, so the hardware prefetch stays within one IOMMU mapping (benign). No cross-buffer overlap → no wild → no drop → recovery never fires. The 3 cores read regions of the one buffer concurrently (benign). Multicore speed is fully preserved.

The one-line blind spot that hid CONTIG for weeks: is_valid_dma_addr (src/npu.c) validated the weight base against w->Bb[]/w->Bf[]/scratch but had no case for w->Bbc. So a valid Bbc.dma+offset base (e.g. 0xfc67c000, inside the 25.7 MB Bbc) was FALSE-flagged "wild/unallocated (outside all valid buffers)" → validate_regcmd refused → ork_dyn_begin_colsplit returned NULL → fell back to the wedging per-slice path. CONTIG's wbase arithmetic (Bbc.dma + sloff + t0·Kp·32) was correct all along. Lesson: when a "wild address" lies inside an expected buffer's range, suspect the validator's known-buffer list before the address math.

256×3584×3584 (the drop-prone shape; the per-slice path drops on iter 0):

• Full shape suite (test_f16colsplit, Sk=1 and Sk=2, various K/N): all BIT-EXACT.
• make test (every mode: matmul/layer/model fp16+int8, i4, stream, SDP, mode-transition): ALL TESTS PASSED.

CONTIG is now default-on for Sn==1 within the colsplit path (ORK_F16_NO_CONTIG opts out for A/B). The fp16 colsplit itself stays behind ORK_F16_COLSPLIT pending the end-to-end A/B-vs-mcworker gate + un-gate.

The recovery protocol built along the way is kept as the safety net for the benign miss class and any future Sn>1 fp16 shape where CONTIG's single-buffer assumption doesn't hold: adaptive sentinel detect (1.5× the measured slice time, ~24 ms vs a flat 800 ms), reap_stuck (per-core nonblock dummy → clean timeout_clean reap), ork_dummy_probe (int8 nonblock host-bounded health-gate — can never hang), reset-first, and the short reap-timeout precondition (submit timeout < detect window, so a dropped job is reapable at reap time). Diagnostics that made the whole thing legible: ORK_F16_TRACE=<file> (line-flushed, tailable step log — survives a hard wedge via off-box netconsole) and ORK_CAMPAIGN (per-iter bit-exact vs nc=1, catches transient wrong answers an end-only check misses).

Prevention beats recovery for a wild-DMA drop. The corruption is done microseconds after the drop, before any userspace detect can react, and every reap path trips a distinct unfixable vendor-kernel bug (abort domain-put-before-reset, the global-domain-refcount double-put, D-state continue_wait). Removing the cross-buffer boundary (CONTIG) removes the wild at its source — one weight buffer, no boundary, no drop, 2.6× preserved, bit-exact. See NPU Quirks — fp16 3-core colsplit drop.