Exp-2026-08-17 — Big+little proportional CPU MoE dispatcher (RK3588): PARITY, HW-defeated

Repo: ggml-ork (ggml-ork.cpp). Status: DEAD on RK3588 (parity, not a win). Preserved:

Patch (in this zip): parent-experiments-stashed.patch (biglittle hunks in ggml-ork.cpp) — git apply onto the banked tree to restore this experiment; the git-native copy is the snapshot branch above.

branch wip/moe-saga-2026-08-18 (llama.cpp-rockchip) — ork_bl_run_range / ork_bl_compute_set / ork_bl_init_backend / ggml_backend_ork_mul_mat_id_biglittle (~ggml-ork.cpp:3248–3413), dispatch at 6777, bl_* ctx fields. Knobs: ORK_MOE_BIGLITTLE, ORK_BL_LITTLE_FRAC, ORK_BL_POLL, ORK_BL_{LITTLE,BIG}_THREADS, ORK_BL_SPLIT.

Hypothesis

RK3588's 4×A55 little cluster could add MoE-prefill throughput if given a proportional, independent partition (its own affinity-pinned ggml threadpool, no shared barrier) — token-split or expert-split, big cluster (A76) + little cluster (A55) concurrent.

Method

Two affinity-pinned ggml CPU backends (A76 cpu4-7 + A55 cpu0-3), independent threadpools; ORK_BL_SPLIT selects token-split (ork_bl_run_range, little gets ORK_BL_LITTLE_FRAC of tokens) or expert-split (ork_bl_compute_set, gather by expert). ORK_BL_POLL tunes the futex handoff. ork claims MUL_MAT_ID and delegates each partition. Bench: 35B mixed.gguf, pp512, -t4.

Result

• Token-split balances at frac≈0.12–0.15 (big≈little wall) — the split works at the op level.
• Net = PARITY: poll=50 frac=0.12 → 26.24 t/s == native 26.26. The A55s add NO net throughput. The earlier poll=0 "+1 t/s" was just masking a futex-handoff cost; removing the handoff (poll=50) also removes the apparent gain.
• Root cause: prefill is DRAM-burst-bound in the fused-GEMM bursts (native peaks DRAM ~96%), and the A55s' ~0.18× marginal compute is offset by per-op thread-spawn / dual-pool concurrency overhead. Consistent with the RK3588 DRAM decode wall. (Corollary: any ork-claims-the-op split also risks the repack-forfeit-on-claim penalty; mitigated but not a win here.)

Conclusion

DEAD on RK3588 (parity). Little cores are not a throughput lever for compute+DRAM-bound MoE prefill on this SoC. Kept as a correct, HW-defeated reference — the token-split framework itself is sound.

Follow-up (only if HW changes)

Viable only on wider-IOVA / DDR5 / more-balanced-cluster hardware, or if paired with a different engine as the second partition (the NPU as a concurrent token-slice is a bigger added engine than the A55s — see the NF4 CPU-prefill win + the int4-expert-coalesce WIP). Resume from wip/moe-saga-2026-08-18. NOT needed for the current RK3588 target.