Exp 2026 07 12 Matmul Shape Boundaries - oRKLLM/ork-driver GitHub Wiki
Commit: ork-driver feat/ssm-npu Β· src/npu.c (ork_bmm_fp16_fused sched fix) Β· probe tools/f16_shape_probe.c
(sweep + ork_f16_fuzz_* register bisect) Β· reproducer tools/ssd_fusedchain_probe.c. Board RK3588 10.3.0.236.
Bare synth()/synth_i8() with sched=1 zeros (or garbles) the output below a K threshold β fp16 at
K < 96 (i.e. Kβ{32,64}), int8 at K < 256. The single load-bearing register is 0x201:0x1040
(the K-reduction schedule): the sched=1 formula base = 177 β 15Β·(K/S β 1) (S=256 fp16 / 512 int8)
overshoots the template default 0xB1 (177) as K drops (0xBC@K64, 0xBE@K32 fp16), selecting an
invalid K-reduction atom β the MAC array emits zeros. The correct paths (ork_mm_run single-core npu.c,
mcworker) never hit this because they compute sched per shape and use sched=0 (which leaves the template
0x1040) below the schedule's valid band. The fused batched-GEMM primitive hardcoded sched=1 and so broke
at the SSD-scan's K=64 stages. Fix: match the run-path sched predicate. Validated bit-exact across the
scan shapes.
ork_bmm_fp16_fused (chains nb real fp16 matmuls into one PC-chained submit for the on-NPU SSD/Mamba-2
scan) produced zero output at K=64 where the per-op primitive ork_bmm_fp16 is correct:
| shape (M,K,N) |
ork_bmm_fp16_fused (was) |
ork_bmm_fp16 (per-op) |
|---|---|---|
| 64, 128, 64 | rel-L2 5.4e-8 β | β |
| 64, 64, 128 | rel-L2 1.000 β zero | β |
| 64, 64, 64 | rel-L2 1.000 β zero | β |
synth()/synth_i8() take a sched flag. It toggles only two registers: 0x1010 (row/CBUF hint) and
0x1040 (K-reduction schedule). At M=64,K=64 0x1010 is identical either way (16Β·(mc+1)), so only
0x1040 differs:
-
sched=1, K=64 fp16:scale=64/256=0.25,base=(int)(177+11.25)=188=0xBCβ0x1040=0xBC. -
sched=0: leaves the REGCMD template default0x1040=0xB1(regcmd_array_4x32x16.hword 0).
The correct fp16 paths compute sched = (K pow2) && Kβ[128,2048) (npu.c single-core run + mcworker), so at
K=64 they use sched=0 β the template 0xB1 β correct. The broken paths hardcode sched=1:
ork_npu_probe_f16_mm, ork_ssd_probe_rawmm_f16, ork_ssd_probe_fusedmm_f16, ork_bmm_fp16_fused.
Register bisect on silicon (f16_shape_probe 1, ork_f16_fuzz_add(0x201,0x1040,Β·)), M=64,K=64,N=64:
0x1040 |
rel-L2 | verdict |
|---|---|---|
0xBC (sched=1 formula) |
1.000 | ZERO-OUT |
0xB1 (template default) |
3.8e-8 | correct |
Overriding only 0x1040 flips zeroβbit-exact. Causation confirmed.
Why the formula overshoots: base = 0xB1 β 15Β·(2^lg β 1) with lg = log2(K/256) is calibrated for the
prefill band (Kβ₯256 fp16 / Kβ₯512 int8, where 2^lg β₯ 1). Extrapolated below the reference K, 2^lg < 1 makes
the correction term negative, pushing base above the template 0xB1. The K-reduction schedule value is
monotonic in K (0xA2@K512 β 0xB1 template β 0xBC@K64); a value too large for the actual K selects a reduction
atom the MAC array can't satisfy for that contraction length β zero output.
fp16 (f16_shape_probe 0, Mβ{1,8,64} Γ Kβ{32..512} Γ Nβ{16..256}) β the boundary is purely K-driven,
N- and M-independent:
| K | fp16 sched=1
|
note |
|---|---|---|
| 32 | ZERO-OUT (M=1) / wrong (M>1) | 0x1040=0xBE |
| 64 | ZERO-OUT / wrong | 0x1040=0xBC |
| 96 | correct | first valid; 0x1040=0xBA
|
| 128β512 | correct | prefill band |
(M=1 zeros; M>1 garbles β same broken schedule, the M=1 all-zero input just makes it read as pure zero.)
int8 (f16_shape_probe 3, M=64,N=64) β same mechanism, wider band (formula scales by K/512):
| K | int8 sched=1
|
|---|---|
| 32, 64, 96, 128 | ZERO-OUT |
| 256, 512, 1024 | correct |
The int8 K=64 case is fully wrong at every tested N (32β¦512) β i.e. it is the K-schedule bug, not an
N-width effect. This reconciles the earlier FLOOR_DECOMP note ("raw-synth_i8 at M=64,K=64,N=1024
computes only 512 of 1024 cols"): the primary defect is the small-K 0x1040 schedule; any partial-N symptom
at N=1024 is that compounded with output-width/N-tiling, and routing through the run path's config fixes both.
0x1040 is a K-AND-rows schedule, not a constant. Forcing the template 0xB1 at large K breaks the other
direction: at M=64 the 0xB1 override goes partial-rows at Kβ₯384 (f16_shape_probe 2: K=384 rel-L2 0.85,
K=512 0.99). This is why the run path selects 0x1040 per (K, mc-group) rather than pinning one value β and why
the fix is "use the run-path sched predicate," not "always use the template."
| path | fp16 valid region | int8 valid region | notes |
|---|---|---|---|
bare synth/synth_i8 (sched=1) |
K β₯ 96 | K β₯ 256 | raw probes; small-K zeros |
ork_mm_run / mcworker
|
all K (per-shape sched + M-tile) |
all K | production; sched=1 only for pow2 Kβ[128,2048) fp16 / Kβ{512,1024} int8 |
ork_bmm_fp16 (β ork_mm_run) |
all K | β | correct at K=64 (goes through run) |
ork_bmm_fp16_fused (chain) |
K β₯ 96 (before) β all scan K (after fix) | n/a | single M-tile (mc=M β€ K-dependent chunk) |
ork_bmm_fp16_fused now computes sched exactly like the run path instead of hardcoding 1:
int sched = (K&(K-1))==0 && K>=128 && K<2048; /* K<96 -> sched=0 -> template 0x1040 (correct) */
synth(rc, M, K, N, ..., sched, CBUF);Validated (f16_fusedchain_check, board, fixed npu.c):
| nb, M, K, N | rel-L2 | verdict |
|---|---|---|
| 8, 64, 32, 64 | 0.0 | correct (was ZERO) |
| 8, 64, 64, 64 | 4.4e-8 | correct (was 1.0) |
| 8, 64, 64, 128 | 4.3e-8 | correct (was 1.0) |
| 4, 64, 96, 64 | 5.0e-8 | correct |
| 8, 64, 128, 64 | 5.4e-8 | correct (unchanged) |
| 8, 64, 128, 128 | 5.4e-8 | correct (unchanged) |
Board healthy throughout (errno=110=0; the dmesg soft reset lines are the probes' deliberate per-call
ACT_RESET on dtype entry, not wedges β gate wedge-detection on errno=110, not dmesg).
The declared shape rules (K%32, N%16 fp16 / N%32 int8) are correct but too loose on the low-K end for
any raw-synth caller: they admit K=32/64, which the raw path miscomputes. Production callers are safe (the
run path handles low K); the constraint belongs on the raw-synth-based primitives. Documented here + in the
regcmd ISA Reference rather than tightening the
public guard (which would wrongly reject the valid ork_bmm_fp16/run K=32/64 path).
# on board, fresh checkout ~/ork-driver-ssm, branch feat/ssm-npu
cc -O2 -Wall -Iinclude -Isrc -pthread tools/f16_shape_probe.c src/npu.c src/soc.c \
src/soc/rk3588.c src/soc/rk3576.c src/neon_activations.c -o f16_shape_probe -lm
sudo env ORK_MM_TIMEOUT=3000 ./f16_shape_probe 1 # causation: 0x1040=0xB1 fixes K=64
sudo env ORK_MM_TIMEOUT=3000 ./f16_shape_probe 0 # fp16 KΓNΓM sweep
sudo env ORK_MM_TIMEOUT=3000 ./f16_shape_probe 3 # int8 K sweep