Date: 2026-06-26 Status: ✅ M1a DONE — the sparse-MoE routed _exps (MUL_MAT_ID) tensors now persist into and reload from a mixed .orkpack (before: backbone + router + dense shared-expert persisted, zero routed experts). Reload-from-disk NPU-expert output bit-identical to a CPU int8 reference. ★ KEY: running real experts on the NPU surfaced the 4 GiB IOVA cap for real — an 8B MoE's int8 experts exceed it, so compute status -1 once ~1909 distinct experts are resident. Commits: ggml-ork c89cf8654 (synthetic per-expert key + persist/load) · ork-driver 6b0d568 (ork_mm_load_i8 rebuilds full-K Bf). RK3588 10.3.0.236 (NPU validate) / box 10.3.0.241 (convert), governors pinned.

M0 proved the per-expert FFN GEMM is viable on the NPU, but flagged the gap: the mixed-.orkpack convert (Layer-Streaming) persisted only the backbone (incl. router + dense shared-expert) — zero routed _exps / MUL_MAT_ID tensors. So every run re-packed the entire expert pool. M1a: persist the routed experts so the pool reloads as int8-resident chain inputs instead of re-packing per run.

• ggml-ork (c89cf8654). A MUL_MAT_ID weight is one tensor holding n_expert slices. Gave each slice a synthetic per-expert key <name>#<e>. At convert, persist all n_expert slices via ork_orkpack_tier. On load, ork_mm_load_i8 / ork_mm_load_i4a8 (read the pre-tiled int8 bytes back) instead of dequant+repack.
• ork-driver (6b0d568). ork_mm_load_i8 rebuilds the full-K Bf, gated K%512==0 && K<=4096, so a disk-loaded int8 expert is a first-class chain input (same envelope as run_chain_i8/run_stream_i8).
• Model: LFM2.5-8B-A1B (264 expert tensors × 8 experts).
• Correctness gate = reload-vs-CPU-int8-ref, NOT the live NPU MoE path: the live MoE path (mode 0) is itself broken on this model (pre-existing, orthogonal to persist). A deterministic prompt is replayed; the disk-loaded NPU expert output is compared to the CPU int8 reference.

• The mixed .orkpack now contains 2112 _exps entries (264 × 8) vs 0 before.
• Reload-from-disk NPU-expert output bit-identical to the CPU int8 reference (889 experts loaded from disk on the deterministic prompt; matched exactly).

An 8B MoE's int8 experts exceed the 4 GiB NPU IOVA window (the rk_iommu 32-bit cap established in Layer-Streaming). When ~1909 distinct experts accumulate resident, the submit returns compute status -1 — tokens are correct up to the cut, then it falls over.

This is NOT a persist bug — it is the wall, now hit by a real workload rather than a synthetic probe. It mandates the rest of the thread:

• resident-all-experts is impossible beyond a tiny model;
• → M2 tile-streaming (cycle expert tiles through the resident window on the prefill path) and M3 hot-expert cache (keep the most-routed experts resident, stream the cold tail) are the required follow-ons, not optional optimizations.

✅ M1a DONE — routed experts persist and reload bit-identically; the expert pool is now a disk-cached, int8-resident chain input. ★ But the same path proved the 4 GiB IOVA cap bites real 8B MoE experts (compute status -1 at ~1909 resident) — confirming the MoE saturation thread must stream (M2) + cache the hot set (M3), not reside the whole pool.

• M1b — routing/dispatch (which experts go NPU vs CPU/GPU per token).
• M2 — tile-streamed prefill through the resident window (the Tier 9 fill chain).
• M3 — hot-expert decode cache — see MoE Routing Skew (the skew measurement that sizes the cache; verdict: VIABLE, ~3.5 GB → ~50% of decode lookups NPU-resident on the 35B target).