The fp16 M envelope, and why 0x1040 was never a "K-reduction schedule"

Board work on RK3588 (10.3.0.236), 2026-08-20. Started as a small correctness chore — Tier 14 A's "ork_f16_mm_run_stream{,_chain} must REFUSE out-of-envelope M, not miscompute" — and ended up overturning the framing of 0x1040 for fp16, fixing two live silent miscomputes, and widening the fp16 M-tile by up to 5.5×.

Landed: ork-driver main 6584e2a. Probes: tools/re/f16_mcap_probe.c (envelope per K per entrypoint), tools/re/f16_k128_probe.c (non-monotonicity + bank columns), tools/re/i8_mcap_probe.c (int8 control).

TL;DR

• 0x1040 (RK_CNA_CBUF_CON0) is a CBUF BANK SPLIT, not a schedule code. v = (WEIGHT_BANK<<4) | DATA_BANK with DBNK + WBNK == 12 at every step, walked one bank data-ward per 64-row group. This confirms and extends the M-fold entry and the ISA reference naming, now measured on the standard (non-mfold) path.
• The fp16 envelope is therefore just banks × 32 KB / row-bytes: sched=0 → 1 data bank → M ≤ 16384/K; sched=1 → 11 data banks → M ≤ 180224/K.
• The mg_max*64 formula fp16 had inherited from int8 is an artifact of a linear fit to the bank encoding. Symbolically it reduces to exactly the bank bound — 165/15 = 11 — so the "11" is not an empirical constant.
• fp16 attains the bound; int8 stops short — RESOLVED: the split walks data-ward as M grows, so WEIGHT_BANK shrinks. int8 keeps its weight CBUF-resident and dies once WBNK no longer fits it; fp16 streams its weight and is unaffected. Closed form for int8 at N=64: M_max = 64·floor(12/w − 1), w = K/512 — exact at K=512/1024/1536/2048. (One corner survives: see Residual gap.)
• K=128 is broken by encoding and its envelope is non-monotonic; only the contiguous prefix (256) is usable.

The measurement

Reference = per-row M=1 submits of the same weight (in-envelope at every K, so it cannot inherit a formula error), cross-checked against a CPU double accumulation (cpu-rel 0–2.7e-08). Deterministic row-varying A — constant rows would hide exactly the cross-row error being hunted. Every ceiling found by an upward scan with ceiling+1 confirmed wrong.

K	sched	old mg_max*64	old R cap	measured	=
256	1	704	128	704	11 banks
512	1	320	64	352	11 banks
1024	1	128	32	176	11 banks
128	1	1472	256	256	broken encoding
64	0	3456	512	256	1 bank
384	0	448	64	42	1 bank
640	0	256	32	25	1 bank
768	0	192	32	21	1 bank
1536	0	64	16	10	1 bank
2048	chain 0	64	16	8	1 bank

360448 B / 11 = 32768 B per bank, matching NVDLA's CBUF (512-bit entries, 32 KB banks → 512 entries/bank). The sched=0 value falls straight out of the template default 0x1040 = 0xb1 = DATA_BANK=1.

The mechanism, seen directly

f16_k128_probe prints the bank fields per mg. At K=256 the progression is unmistakable:

mg   v      DBNK  WBNK        mg   v      DBNK  WBNK
1    0xb1   1     11          7    0x57   7     5
2    0xa2   2     10          8    0x48   8     4
3    0x93   3     9           9    0x39   9     3
4    0x84   4     8           10   0x2a   10    2
5    0x75   5     7           11   0x1b   11    1   <- ceiling 704, all OK
6    0x66   6     6           12   0x1b   11    1   SAT -> breaks at row 704

DBNK + WBNK == 12 throughout. The v ≥ 0x1b floor is physical, not arbitrary: it is DBNK ≤ 11 ⟺ WBNK ≥ 1. Forcing v = 12 (WBNK = 0) via ORK_R1040 hung the submit — see the board-ops note below. Independent corroboration: i8/regcmd.c's own ORK_R1040 comment already called it a "bank alloc" and cites rknn's captured 0x75 → DBNK=5, WBNK=7, sum 12.

K=128: the encoding cannot express half-bank steps

slope = (int)(15·K/256) = (int)7.5 = **7** and base = (int)184.5 = **184**, so the very first split is DBNK=8, WBNK=11 — sum 19, already invalid — and subtracting 7 per step wraps DBNK mod 16 instead of moving one bank. The envelope is consequently non-monotonic:

M=256 OK · 320 BAD · 384 OK · 448 ALL-ZERO · 512 OK · 576 ALL-ZERO · 640 OK · 704 BAD
768 OK · 832-960 BAD · 1024 OK · 1088-1152 BAD · 1216 OK        (20 ok / 28 bad, 6 recoveries)

A row cap or an area cap can only be monotonic, so both are ruled out. Only the contiguous prefix (256) is usable. mg ≤ 4 works by luck.

Two live miscomputes this found

1. ork_f16_mm_run_stream miscomputed at non-pow2 K. Its doorbell guard was M*K > 32768 — the int8 byte budget applied at 2 B/elem, so 2× too loose. K=384 real ceiling 42, guard permitted 85; K=640 → 25 vs 51; K=768 → 21 vs 42.
2. ork_f16_mm_run — the main public fp16 matmul — miscomputed at K=128 for any M ∈ [257,1472]. The multicore/colsplit path's 4*R gave a 1024-row tile against a real ceiling of 256.

Why the wrong number survived so long

tools/silu_f16_check validated fp16 at 320 and 384 only — never 352. The true boundary sat between the two probe points, so the int8 formula appeared to carry over. The lesson is in the methodology, not the arithmetic.

Methodology warnings (both cost real time here)

• Never bisect this predicate. It is non-monotonic, so bisection lands on an arbitrary valid point. f16_k128_probe PASS 2 bisected K=128/N=64 to "768" when the contiguous ceiling is 256. Scan upward.
• Never derive an fp16 tile from an int8 constant. Every wrong cap here came from reusing an int8 element/byte budget at 2 B/elem.

int8: measured, unchanged, and NOT to be unified

Predicted (wrongly) that int8 would gain the same way. It does not — tools/re/i8_mcap_probe.c, bit-exact vs a CPU int32 reference:

K	mg_max*64	bank model predicted	measured
512	704	704	704 (705 hangs)
1024	320	352	320 (321 mismatches)
2048	128	176	128 (129 hangs)
4096	64	88	64 (65 hangs)

int8 is already correct at its truncated mg_max*64, reproducing Weight-DMA Amortization exactly. Do not move int8 onto the fp16 rule — and the reason is now known, not a coincidence: see the resolution below.

RESOLVED (2026-08-20, later the same session) — int8 keeps its weight CBUF-RESIDENT; fp16 STREAMS it

The divergence is not about groups, capacity, or precision arithmetic. It is about who else is using the banks.

DBNK + WBNK = 12, and as M grows the split walks data-ward — so WEIGHT_BANK shrinks. Writing wd = data banks per 64-row group and ww = banks the weight needs:

wd = 64*K*elsz / 32768      (int8: K/512 ;  fp16: K/256)
ww = K*N*elsz / 32768
DBNK(mg) = mg * wd          WBNK(mg) = 12 - DBNK(mg),  floored at 1 (v >= 0x1b)

• fp16 streams its weight, so only DBNK binds: it runs until DBNK saturates at 11 → M ≤ 11*32768/(K*2) = 180224/K.
• int8 holds its weight resident, so it additionally needs WBNK ≥ ww. It dies at the first mg where the shrinking weight bank no longer fits the weight.

At N=64 the two coincide (wd == ww == K/512), which gives the closed form M_max = 64 * floor(12/w - 1) — and it reproduces every int8 measurement exactly:

K	w = wd = ww	WBNK(mg) = 12 - mg*w	last mg with WBNK ≥ w	predicted	measured
512	1	11,10,…,1	11	704	704 ✓
1024	2	10,8,6,4,2,(1)	5	320	320 ✓
1536	3	9,6,3,(1)	3	192	192 ✓
2048	4	8,4,(1)	2	128	128 ✓

Why the dtypes appeared to agree at one point: when the weight is exactly 1 bank, the clamp's WBNK=1 is still sufficient — so int8 K=512 and fp16 K=256 both reach the full 11 data banks (704). They diverge everywhere else because only int8 is competing for the banks.

Evidence fp16 does not compete: its ceilings 704/352/176 at K=256/512/1024 are all exactly 11 data banks, with weights of 1, 2 and 4 banks respectively. Weight size does not move the fp16 ceiling.

Evidence int8 does: at K=1536 (3-bank weight) the natural walk is bit-exact at M=192 (WBNK=3) and every element is wrong at M=193, the first mg where the clamp drops WBNK to 1. Forcing v=0x1b at K=1536 breaks it even at M=128, far inside any capacity bound — the weight has nowhere to live.

Falsifiable prediction (untested): since ww scales with N, the int8 ceiling should be N-dependent — doubling N to 128 halves ww's headroom and should roughly halve the ceiling. The fp16 ceiling should not move at all. One N sweep would settle it.

The hypothesis this replaces (refuted)

The earlier guess — "the host A-staging layout is a function of DATA_BANK" — is wrong, killed by a datum already in hand: int8 K=1024 M=256 is bit-exact at its natural DBNK=8 and with DBNK forced to 11. Same staged A, two different bank counts, both correct. A layout that tracked DBNK could not do that.

Can the pool be stretched past 12? NO — measured in both directions

The obvious follow-on: NVDLA's CBUF is 16 banks, DBNK/WBNK are 4-bit fields, and there is a third field FC_DATA_BANK[10:8] (always 0 in our templates). So is the usable pool really 12, or are we leaving banks on the table? Tested by forcing over-committed splits with ORK_R1040:

forced v	DBNK	WBNK	sum	int8 K=1024	result
0x2a	10	2	12	M=320	OK, bit-exact (the natural value)
0x2b	11	2	13	M=352	22471/22528 elements wrong (99.7%)
0x0c	12	0	12	M=352	submit HANGS (wedges core 0)

The pool is 12, and both edges are hard. WBNK cannot go below 1 (hang), and DBNK+WBNK cannot exceed 12 (garbage). Note the sum-13 case is worse than the natural clamp at the same M — 99.7% wrong versus 55% — which is the signature of an invalid allocation rather than an insufficient one. So fp16's 11 data banks is the hard maximum: 12 total, minus the one bank the weight must always hold.

(Validity of the override is established independently: at K=1536 M=128 the natural v=0x66 is bit-exact while a forced v=0x1b breaks every element — same shape, different result, so ORK_R1040 demonstrably takes effect. The 0x2a row above is NOT proof of that, since 0x2a is what mg=5 programs anyway.)

Is there reclaimable slack inside a partly-used bank? NO

DBNK[3:0] / WBNK[7:4] are integer bank counts — a bank belongs to data or to weights, and there is no field to express a fractional split, so space left over inside the weight's last bank is unreachable. At N=64 there is no slack to reclaim anyway: the packing is exact (K=1536 → 9 data banks = 294912 B for 192×1536 = 294912 B, and 3 weight banks = 98304 B for a 98304 B weight). Slack appears only when ww is non-integral — e.g. K=512/N=96 needs 1.5 banks, rounds to 2, and strands 16 KB (~5% of the achievable M) — and it is not recoverable at this granularity. "Rounding up" is not an option either: using more rows than the allocated banks hold is exactly what miscomputes.

Residual gap

K=4096 int8 does not fit the model: ww=8, so floor(12/8 - 1) = 0, yet M=64 (mg=1, WBNK=4) works. With 8 data + 8 weight banks > 12 the weight cannot be fully resident there, so int8 must be streaming (or K-slicing) it at large K — but it then stops at 64 rather than the DBNK-bound 88. The "whole group vs fractional group" question survives only in that corner.

Board-ops: how this wedged the board

Forcing ORK_R1040=12 (WEIGHT_BANK=0) hung the submit, and the external timeout 45 wrapped around the probe then killed it mid-submit — the IOMMU-wedging path AGENTS.md warns about. NPU stuck at 99% with no owning process; the box went unresponsive to ping and SSH. A "Rock 5B Plug" power cycle recovered it (back in 18 s, 0 faults, no SPI corruption). Repin the DDR governor afterwards — it returns as dmc_ondemand.

Rule: when probing deliberately-invalid register space, rely on ORK_MM_TIMEOUT so the driver reaps its own job. Never wrap a live submit in an external timeout/kill.

Validation

make test ALL TESTS PASSED (37 tests) with goldens unchanged ⇒ NPU output byte-identical; check-registry and check-attest clean; tests/sbc_attest.txt refreshed in the same commit.