Exp 2026 10 12 INT4 BCHAIN Dispatch Envelope - oRKLLM/ork-driver GitHub Wiki
Board 10.3.0.235 (ROCK 5B+, LPDDR5). Kernel 6.1.115-vendor-rk35xx-fix.
DMC performance, cur_freq read back as 2400000000 at every arm. CPU performance.
Tool tools/re/i4_storm_probe.c, full-random data (dm=1), ORK_I4_NOLADDER=1, every arm graded
by CHECKSUM against the H=2 anchor for its (K,N,M) — rc and timing are both non-tests.
Every arm ran in its own process behind tools/util/npu_guard.sh.
Supersedes the framing of Exp-2026-10-10 ("the envelope has a DRAM-clock term", "lower H is safer"). Both of those observations stand as measurements; the model built on them did not.
| wall | violated by | signature |
|---|---|---|
activation / H |
H > DBNK·16384/K |
silent miscompute — normal wall time, wrong checksum |
weight tile / Wb |
see §3 |
doorbell miss — submit_rc=0 on every core, output cells never written, ~16 s of recover |
They are independent, and only the first is about H. Conflating them is what produced the
non-monotone "H envelope": at K=512 the shipped Wb=256 fails at H=2 and H=8 but passes at
H=16/32, which is impossible for any ceiling and perfectly ordinary once you notice that the
failing arms are the Wb wall and the passing ones are not.
Confirmed in both directions (ORK_I4_DBNK moves the CBUF DATA_BANK/WEIGHT_BANK split in
0x1040 and the ceiling with it):
| K | DBNK=1 | DBNK=2 | first bad H miscomputes? |
|---|---|---|---|
| 512 | H=32 ok | H=64 ok | yes |
| 1024 | H=16 ok, 32 miscomputes | H=32 ok, 64 miscomputes | yes |
| 4096 | H=4 ok | — | — |
| 8192 | H=2 ok, 4 miscomputes | — | yes |
DBNK=2 is therefore a real, bit-exact doubling of the batching factor at K ≤ 1024. It is not
shipped: see §5.
N=2048, M=128, DMC 2400. PASS = checksum identical to the shape's anchor. miss = doorbell
miss (~3.8 s with ORK_MISS_TO_US=20000; ~16 s with production timeouts). calc = silent
miscompute at normal speed.
| K | Wb | H=2 | H=4 | H=8 | H=16 | H=32 |
|---|---|---|---|---|---|---|
| 2048 | 64 | PASS 4188 | PASS 3588 | miss | miss | miss |
| 2048 | 128 | miss | miss | miss | miss | miss |
| 2048 | 256 | miss | miss | miss | miss | miss |
| 1024 | 64 | PASS 3281 | PASS 2620 | PASS 2368 | PASS 2235 | calc |
| 1024 | 128 | miss | miss | miss | miss | calc |
| 1024 | 256 | miss | miss | miss | miss | calc |
| 512 | 64 | PASS 2691 | PASS 2106 | PASS 1903 | PASS 1785 | PASS 1720 |
| 512 | 128 | PASS 1976 | PASS 1645 | PASS 1499 | PASS 1453 | PASS 1448 |
| 512 | 256 | miss | miss | miss | PASS 1257 | PASS 1300 |
DBNK=2 column at Wb=64: K=2048 H=4 PASS / H≥8 miss (unchanged); K=1024 H=4..32 PASS, H=64
calc; K=512 H=4..64 all PASS.
Extension past the shipped Wb floor (Wb = (131072/K)&~63 clamps at 64 for K ≥ 2048):
| K | Wb | K·Wb | H=2 | H=4 |
|---|---|---|---|---|
| 4096 | 64 | 262144 | PASS 6291 | PASS 5301 |
| 8192 | 64 | 524288 | PASS 10282 | calc |
| 256 | 256 | 65536 | — | PASS (H=32) 1308 |
Wb == 64 -> ALWAYS dispatches (26/26 points, K = 512 … 8192)
Wb > 64 and K·Wb <= 65536 -> ALWAYS dispatches (K=512/Wb=128, K=256/Wb=256)
Wb > 64 and K·Wb == 131072 -> ERRATIC, K-dependent (K=2048 only H<=4; K=1024 none; K=512 only H>=16)
Wb > 64 and K·Wb >= 262144 -> ALWAYS misses (10/10 points)
65536 int4 elements is 32768 bytes — one CBUF bank (12 banks × 32 KB). So:
-
the shipped
Wb = 131072/Kaims at TWO banks. The comment ini4/chain.ccalling 131072 elements "one CBUF weight bank" is off by a factor of two, and the value it picks is precisely the erratic regime. -
Wb = 64meansNBc = Wc/64 = 1, i.e. a single 64-column output block per program. That case never misses at anyKtried, includingK·Wb = 524288(K=8192).
Measured by collapsing the chain to a single program (N = Wb so NC=1, M = H so NG=1), with the
Wb=64 run of the same (K,N,M,H) as the anchor:
| K=2048 N=128 H=4 Wb=128, NBc=2 | NT=1 | 2 | 4 | 8 | 16 | 32 |
|---|---|---|---|---|---|---|
| result | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL |
NT=1 already fails, identically at every chain length, and at NBc=4 and at K=1024 too. The single
program's failure is a clean doorbell MISS, not a miscompute: rc=-1, 3.76 s, and sat+=128 —
every one of the one program's 128 written cells still at the seed. Nothing landed.
The K boundary at Wb=128 is then sharp and exactly on the bank (N=128, M=H=2, NC=NT=1):
| K | 256 | 512 | 768 | 1024 | 1536 | 2048 |
|---|---|---|---|---|---|---|
weight bytes/program = K·Wb/2
|
16384 | 32768 | 49152 | 65536 | 98304 | 131072 |
| result | PASS | PASS | MISS | MISS | MISS | MISS |
(the two PASS checksums are bit-identical to their Wb=64 anchors, 132d2f400fb886bb and
805767767d9ec7c9.)
So the law is:
NBc == 1 -> no weight-tile limit at all (K=8192/Wb=64 = 256 KB of weight per program runs fine)
NBc >= 2 -> the program's weight tile must fit ONE 32 KB CBUF bank: K·Wb/2 <= 32768
which is a capacity law, not a race: a multi-block program presumably loads its whole Wc-wide
weight tile into one weight bank and iterates the 64-column blocks out of it, so a tile that does not
fit never completes its load and the sequencer never fires. A single-block program streams its weight
sequentially and consumes it as it arrives, so it has no residency requirement. The one cell that does
not fit this picture is K=512/Wb=256 (two banks), which fails at H≤8 and passes at H≥16 — a race
sitting on top of the capacity boundary.
Consequence: the multi-block wall is NOT designable-around by pacing, ordering or chain shaping.
The only way to use Wb > 64 at K >= 768 is to make K per program smaller, i.e. K-slicing — see
§6a for why that does not pay.
Everything below is about the Wb=64 / NBc=1 failure at K=2048, H>4 — a different wall from
§3a, with the opposite signature. Three independent pieces of evidence, all at
K=2048 N=2048 M=128 H=8 Wb=64:
-
The job is not "never started".
ORK_MC_DIAG's census reports ~36% of the written cells still at the seed, scattered over 483 runs spanning the whole surface (first=1024,last=317471), withsubmit_rc=0on every core. Most programs in the accepted chain ran; individual ones dropped. A capacity violation cannot drop program #200 and run #201. - Fewer cores makes it WORSE: dropped fraction nc=3 36% → nc=2 43% → nc=1 49%. So it is not inter-core contention for DRAM or for the IOMMU; it tracks chain length within one core.
- A lower DRAM clock closes it (1320 MHz passes 10/10 bit-identical) and a reset makes the identical job return bit-exact. Capacity laws are indifferent to DRAM clock.
Each measured at K=2048 N=2048 M=128 H=8, each left the miss unchanged:
| knob | result |
|---|---|
ORK_I4_WREUSE=0 (drop CBUF weight-reuse, 0x1040[13]) |
miss |
ORK_NPU_MC=1 / =2 (serialize the cores) |
miss, and WORSE drop fraction |
ORK_MOE_SRAM_SCRATCH=1 (regcmd/activation/task/output into the 956 KB on-chip NPU SRAM) |
miss |
ORK_I4_DBNK=2 (double the CBUF data banks) |
miss |
ORK_I4_BURST sweep (RK_CNA_DMA_CON0) |
miss — and reducing burst breaks the H=4 that works |
| NPU devfreq 1000 → 400 MHz (earlier) | miss at every step |
ORK_NO_PINGPONG=1 (submit flags 0x1 instead of 0x5) |
miss (H=4 stays correct, so the knob reached the hardware) |
ORK_I4_1010 (CNA CONV_CON2) |
miss — and it corrupts H=4 too, so unlike int8 this register is NOT an inert perf hint on the int4 path; do not use it as a timing nudge |
On the SRAM lead specifically: the prior "separate port, NULL for NPU weight reads" finding was a bandwidth measurement and did not rule this out, which is why it was retested. It is now ruled out for this failure: staging the whole per-program working set (regcmd, activation, tasks, output) on the second port changes nothing about whether the chain's programs land.
At Wb=64, every K reaches its full activation budget H·K = 16384 — K=512 at H=32, K=1024
at H=16, K=4096 at H=4, K=8192 at H=2 — except K=2048, which stops at H=4 (H·K = 8192).
That is one isolated K, not a monotone law, and it is the K production uses most.
It is a property of the whole K=2048 row, not of one shape: H=8 miscompares against the H=4
anchor at N = 512 / 1024 / 2048 / 4096 (M=128) and at M = 64 / 128 / 256 (N=2048) — eight
shapes, zero exceptions.
The geometry ladder (added 2026-10-10) recovers from these misses bit-exactly. But nothing recorded that the part had already refused a geometry, so the refused attempt was re-paid on every call:
shape (N=2048 M=128) |
before | after | warm speedup |
|---|---|---|---|
| K=2048 | 15 860 µs | 3 435 µs | 4.6× |
| K=1024 | 15 857 µs | 2 141 µs | 7.4× |
| K=512 (never missed) | 1 301 µs | 1 249 µs | — |
Fix (i4: remember the BCHAIN rung this part accepts, and probe for it cheaply):
-
orki_i4_run_bchain_dbmoved tosrc/npu/i4/run.cand given a per-part(K,N) → rungmemo that only ever tightens — a lower rung is bit-exact, and the de-escalation path goes toWb=64first, the one N-tile width every measuredKaccepts. A part that refuses nothing (.236, where theorki_i4_hcaptable was measured) never writes an entry and pays nothing. -
ladder_probeonork_dyn_chain: an attempt the caller can fall back from skips the six recover rounds (resetting and resubmitting a geometry this part will not take cannot make it accept) and caps its completion wait at 1 s. Learning a shape costs ~1.33 s instead of 15.9 s.
Two traps found while fixing it, both worth remembering:
- the reset before a retry must be conditioned on having missed, not on "we are past rung 0".
Resetting on a memoized start costs a full
ACT_RESET+ cold re-warm — measured 216 ms per call, which ate 98% of the win and looked exactly like the memo not working. - a rung that is not a different geometry from the one above it must be skipped, or a shape records a rung it can never reach and every later call skips the attempt that works.
make test ALL PASS on .235, tests/sbc_attest.txt refreshed.
| shape | ladder lands on | best measured | gain |
|---|---|---|---|
| K=2048 N=2048 M=128 | Wb=64 H=4 — 3435 µs | same | 0 |
| K=1024 | Wb=64 H=16 — 2141 µs | Wb=64 H=32 DBNK=2 — 2122 µs | 1% |
| K=512 | Wb=256 H=32 — 1249 µs | Wb=256 H=16 — 1257 µs | 0 |
Wall time here is floor(M,N) + NT × ~2 µs, with NT = (N/Wb)·ceil(M/H)/nc and a host-side floor of
roughly 2 ms at M=128 N=2048 (seed + DMA sync + verify + de-tile of a ~700 KB tiled surface). So
H and Wb are the same lever — only their product matters — and at these shapes the floor, not
the batching factor, is what dominates. Raising H further is not where the remaining time is.
Making DBNK=2 the default would require re-measuring all 25 orki_i4_hcap entries, since the
activation ceiling moves with it, for ~1%. Changing the default Wb to the one-bank value would
help .235 and probably hurt .236 (fewer, wider tiles = fewer programs there), and .236 was not
available to check. Both are left alone; the memo makes the right choice per part at run time.
Phase-timed in-tree (temporary instrumentation, reverted), K=2048 N=2048 M=128 H=4 Wb=64 nc=3,
NT=320/core, need_o=655360 B/core, total 3628 µs, worst core:
| phase | µs | on the critical path? |
|---|---|---|
| A pack (memset + nibble tile + maf sync) | 87 | yes |
| regcmd build (320 synths + mrc/mtk sync) | 203 | yes |
| SENT16 seed (written cells only) | 71 | yes |
| seed sync (mcc TO_DEVICE, padded extent) | 58 | yes |
| submit ioctl | 13 | yes |
| poll loop | 2905 | the NPU wait |
| — of which mode-1 gate | 326 | no, hidden |
| — of which FROM_DEVICE sync (padded) | 46 | no, hidden |
| — of which mode-3 verify | 233 | yes (runs after completion) |
| end invalidate (padded, and the poll already did it) | 64 | yes |
| de-tile + writeback | ~148 | yes |
So: NPU execution ≈ 2300 µs (63%), host critical path ≈ 910 µs (25%), not the 59% floor the padding question assumed.
The padding verdict. The seed, the drain-verify and the de-tile all already walk only the written cell set — a previous agent fixed those. What still runs over the full padded slot is the three DMA cache-maintenance syncs: 58 + 46 + 64 = 168 µs, 4.6% of the call, of which only the 58 and the 64 are on the critical path and only ~¾ of those is recoverable. Best case ≈ 90 µs, 2.5%. The hypothesis is correct and small. Not taken.
Poll-window host work is free. ORK_I4_PREDICT=1 removes the mode-1 gate almost entirely
(326 → 7 µs) and the total does not move: 3422/3544/3553 (off) vs 3514/3531/3585 (on), interleaved
ABABAB to cancel this board's arm-position bias. That both explains the old "predictive drain measured
no effect" result and tells us the gate is not stealing DRAM bandwidth from the NPU.
Two independent separations agree. Varying H moves only NT (activation traffic is constant), and
gives 1.7 µs per program. Varying Wb at K=512 moves both; subtracting the per-program term
leaves 366 µs for 524 KB of extra activation, i.e. ~1.43 GB/s. Applying that rate to K=2048's
2.62 MB/core of activation predicts 1.76 ms, which is exactly the NT-independent part of the measured
curve (~1.49 GB/s).
So at production K the dominant single cost in the entire call — about half of it — is the NPU
re-reading the whole M×K activation once per N-tile, at ~1.45 GB/s. With Wb=64 forced, K=2048
has NC=32 tiles, so the activation is read 32 times over.
That makes larger Wb, not larger H, the performance direction — and §3a says larger Wb is a
hard capacity wall at K≥768. The measured ordering confirms the direction: at K=512, Wb 64 → 128 →
256 runs 1720 → 1448 → 1257 µs at matched H·Wb.
K-slicing to get there does not pay. Splitting K=2048 into 4×K=512 at Wb=128 (one bank, legal)
cuts NT 320 → 96 and activation 2.62 → 1.57 MB/core, worth roughly 870 µs of NPU time — but it
multiplies the per-slice host critical path (seed, verify, de-tile, syncs) by four, about +900 µs.
A wash at best, and that is before the partial-sum accumulate.
Half the activation bytes are zeros. The int4 batch mode is stride-2: mc_phys = 2·H, logical row
m at physical row 2m, with the odd rows memset to zero. If the hardware reads them — and the
traffic arithmetic says it does — then ~25% of the whole call is the NPU streaming zero rows. Whether
a stride-1 activation layout is expressible (the 0x405c axis) is the open question with the largest
payoff found here, and it was not tested.
-
Can the activation be stride-1? The
0x405c = 0stride-2 batch layout means half the bytes the NPU reads per program are the zeroed odd rows, and activation re-reads are ~50% of the call at K=2048. A stride-1 variant would be worth ~25% — far more than anything else found here. Needs a0x405c/DPU_WDMAregister probe plus a matching de-tile. -
Raise
Wbat large K by shrinking the per-program weight tile another way. §3a makes this a one-bank capacity law, so the lever is K per program, and plain K-slicing is a wash (§6a). A K-sliced chain that shares ONE seed/verify/de-tile pass and accumulates in the de-tile would keep the 870 µs of NPU saving without paying the host multiple; that is the shape of the win. - The residual K=2048 / H>4 hole (§4) is worth ~270 µs if closed, and remains unexplained.
- NPU clock, DMA burst length, SRAM staging, weight-reuse, core count — all measured, all null for this failure (§4).
- Underclocking DRAM as a remedy: 1320 MHz + H=8 is 4183 µs against 2400 MHz + H=4 at 3503 µs. ⚠ RETIRED 2026-10-13 — does not reproduce on the current tree (1320+H=8 now 3326–3366 µs vs 2400+H=4 3397–3445, i.e. the de-rate is 1–3.5% FASTER; the Wb/ladder and completion-gate work closed it). See INT4-BCHAIN-Operational-Notes § "the clock term is a STEP, not a slope".
- A per-die calibration table or a clock-aware geometry formula. There is no monotone ceiling to fit; the memo gets the same answer at run time with no table to maintain.
-
Making
Wbexpress 32 columns. REFUTED as a lever by §3a:Wb=64 is alreadyNBc=1 and therefore already exempt from the weight-tile capacity law at any K, so a 32-column tile unlocks nothing — and the direction that actually pays isWblarger, not smaller (§6a). - Chain pacing, reordering or inserting no-op tasks to relieve the multi-block wall. NT=1 fails, so there is no chain to pace.
- Narrowing the padded DMA syncs: measured, 2.5% best case (§6a). Available if someone wants it, but it is shared drain code and the int4 coherency path has a history.
- Removing host work from inside the poll window: measured free (§6a).
Board 10.3.0.236. Kernel 6.1.115-vendor-rk35xx-fencefix. DMC performance, cur_freq
read back as 2112000000 (its MAXIMUM — this part has no 2400 OPP) at every arm. CPU cpu4/cpu6
2400000. Library a clean GitHub clone at 62c5b47 in ~/work/orkmap, isolated from the shared
board tree. Tool tools/re/i4_storm_probe.c, full-random data (dm=1), ORK_I4_NOLADDER=1,
ORK_MISS_TO_US=20000, every arm its own process behind tools/util/npu_guard.sh, every arm graded by
CHECKSUM against the Wb=64 H=2 anchor for its own (K,N,M).
The method was first validated on .235, where it reproduced §2's K=512 row and §3a's Wb=128
K-ladder exactly, checksums included.
N=2048, M=128. Scanning H upward at Wb=64:
| K | largest PASS H | H·K | first bad H | its signature |
|---|---|---|---|---|
| 512 | 32 | 16384 | 64 | miscompute |
| 1024 | 16 | 16384 | 32 | miscompute |
| 1536 | 11 | 16896 | 12 | miscompute |
| 2048 | 8 | 16384 | 16 | miscompute |
| 2560 | 4 | 10240 | 5 | miss (storms) |
| 3072 | 4 | 12288 | 5 | miss |
| 4096 | 4 | 16384 | 8 | miss |
| 5120 | 3 | 15360 | 4 | miscompute |
| 6144 | 3 | 18432 | 4 | miscompute |
| 8192 | 2 | 16384 | 4 | miscompute |
Zero doorbell misses at Wb=64 at any valid H, at any K from 512 to 8192 — so §3's
Wb == 64 -> ALWAYS dispatches clause holds on a second die, a different memory type and a different
kernel. Every entry reproduces orki_i4_hcap (which was measured on this part), and K=2048 takes
H=8 here, where .235 stops at 4 — the difference that motivated this mapping.
.236's residual holes are at K=2560 (H=4 where 16384/K allows 6) and K=3072 (4 where it
allows 5), not at K=2048. So both parts have holes in the "every K reaches H·K = 16384" picture;
they are simply at different K.
§3a's one-32-KB-bank law reproduces perfectly on .236 along the ladder it was measured on — same
PASS/MISS pattern at Wb=128, and the passing checksums are bit-identical across the two boards
(132d2f400fb886bb at K=256, 805767767d9ec7c9 at K=512). Off that ladder it fails in three
independent ways.
(a) Equal bytes land on both sides. Single program throughout (N=Wb, M=H=2, so NC=NG=NT=1):
| K | Wb |
K·Wb/2 bytes |
result |
|---|---|---|---|
| 512 | 192 | 49152 | PASS |
| 768 | 128 | 49152 | MISS |
| 704 | 128 | 45056 | MISS (intermittent, see §11) |
| 256 | 768 | 98304 | PASS |
| 640 | 192 | 61440 | MISS |
K=256 passes at 131072 bytes (Wb=1024, NBc=16) — four full banks — while K=640 misses at
61440. No byte budget separates these.
(b) The CBUF bank split does not move it. K=512 Wb=256 (a MISS) re-run with ORK_I4_1040 =
0xa2, 0x93, 0x84, 0x66, 0x48 (WEIGHT_BANK 10, 9, 8, 6, 4) — all still miss; and
K=512 Wb=192 (a PASS) at 0x84/0x48 still passes. A capacity law in the weight banks would
have to move with the number of weight banks. It does not move at all.
(c) ★ Raising H rescues the identical weight tile. N=Wb and M=H, so NT=1 at every row and
the per-program weight tile is byte-for-byte the same:
| K | Wb | H=2 | H=4 | H=8 | H=16 |
|---|---|---|---|---|---|
| 768 | 128 | MISS | PASS | PASS | PASS |
| 1024 | 128 | MISS | MISS | PASS | PASS |
| 512 | 256 | MISS | 2/3 ERRATIC | PASS | PASS |
| 2048 | 128 | MISS | MISS | MISS | (miscompute — over the H ceiling) |
A residency requirement cannot be satisfied by computing more rows out of the same resident tile.
§3a's "NT=1 already fails" is correct about chain length and was measured at M=H=2 — the shortest
compute per weight load there is, i.e. the worst corner of a race.
Wb == 64 (NBc == 1) -> always dispatches; H bounded ABOVE only, by orki_i4_hcap(K)
Wb > 64 (NBc >= 2) -> H must ALSO exceed a floor H_race(K, Wb) that RISES with K
and the geometry is usable only where H_race <= orki_i4_hcap(K)
Measured H_race on .236 at Wb=128: K=768 -> 4, K=1024 -> 8, K=2048 -> >8. Against
hcap = 22 / 16 / 8, the windows are [4,22], [8,16] and empty. That is why Wb>64 is
unusable at K=2048 on this part: the H that would close the race is above the activation ceiling.
It also explains the shipped Wb = (131072/K)&~63 clamping to 64 exactly from K=2048 up.
The erratic band is the H_race transition zone, and it is narrow:
| arm | reps | PASS |
|---|---|---|
K=704 Wb=128 H=2 |
6 + 6 | 1/6, then 2/6 |
K=320 Wb=384 H=2 |
3 | 2/3 |
K=256 Wb=1280 H=2 |
3 | 2/3 |
K=512 Wb=256 H=4 |
3 | 2/3 |
A 3-rep arm cannot tell 1/6 from 0/n; several of this page's MISS cells on both boards are
therefore "mostly misses" rather than "never dispatches".
.236's shipped geometry is not in that band. 20 reps each, N=2048 M=128, checksum-graded:
| K | shipped Wb
|
shipped H
|
result | med |
|---|---|---|---|---|
| 512 | 256 | 32 | 20/20 PASS | 1268 µs |
| 1024 | 128 | 16 | 20/20 PASS | 1638 µs |
| 2048 | 64 | 8 | 20/20 PASS | 2936 µs |
| K | Wb |
.235 @2400 |
.236 @2112 |
|---|---|---|---|
| 512 | 128 | PASS H=2..32 | PASS H=2, 8, 32 |
| 512 | 256 | miss H≤8, PASS H≥16 | miss H=2, PASS H=8, PASS H=32 |
| 1024 | 128 | miss H=2..16, calc H=32 | miss H=2, PASS H=8, PASS H=16, calc H=32 |
| 1024 | 256 | miss H=2..16, calc H=32 | miss H=2, PASS H=8, calc H=32 |
| 2048 | 64 | PASS H≤4, miss H≥8 | PASS H≤8, calc H=16 |
| 2048 | 128 | miss, all H | miss H=2, 8; calc H=32 |
| 2048 | 256 | miss, all H | miss H=2, 8; calc H=32 |
.236's envelope is uniformly the larger of the two — every cell where they differ, .236
tolerates a smaller H or a larger H. Its DRAM runs slower (LPDDR4X 2112 vs LPDDR5 2400), which is
the direction §4 and Exp-2026-10-10 already established for the Wb=64 hole, now seen on the
NBc>=2 wall as well. The two parts are the same law at two points on a clock axis, not two laws.
.236's DMC maximum IS 2112, so unlike .235 it can only go down. userspace governor,
cur_freq read back at every arm and re-read after each call; single program (N=Wb, M=H=2),
6 reps, checksum-graded. PASS count out of 6:
| arm | 2112 | 1560 | 1068 | 528 |
|---|---|---|---|---|
K=672 Wb=128 |
6/6 | 6/6 | 6/6 | 6/6 |
K=704 Wb=128 |
2/6 | 6/6 | 6/6 | 6/6 |
K=736 Wb=128 |
0/6 | 4/6 | 6/6 | 6/6 |
K=512 Wb=256 |
0/6 | 1/6 | 6/6 | 6/6 |
Monotone, on LPDDR4X, at the worst H. The envelope grows as the DRAM clock falls, exactly as
.235's did, and the boundary moves through the erratic band rather than snapping — a race, not a
capacity edge. Together with §12 (the slower-DRAM board having the larger envelope at matched
geometry) this makes the DRAM-clock term a property of the design, not of one die or one memory type.
Diagnostic only. ⚠ The net-loss figure here is RETIRED (2026-10-13) — re-measured on the
current tree the de-rate is 1–3.5% FASTER, not 19% slower, and the envelope is FLAT from 1968 to 2400
(a step, not a slope). See INT4-BCHAIN-Operational-Notes. Original text:
Underclocking measured a net loss on .235 (1320 + H=8 = 4183 µs against
2400 + H=4 = 3503 µs) and nothing here changes that.
ORK_BCH_DEBUG on a ladder-ON run prints the geometry every BCHAIN call actually used:
[bch] K=512 N=2048 M=128 H=32 Wb=256 NC=8 NG=4 nc=3 NTmax=12
[bch] K=1024 N=2048 M=128 H=16 Wb=128 NC=16 NG=8 nc=3 NTmax=48
[bch] K=2048 N=2048 M=128 H=8 Wb=64 NC=32 NG=16 nc=3 NTmax=176
Every call is the TOP rung. Nothing de-escalates, so orki_i4_geomemo_set is never reached and the
memo stays empty. Ladder-ON vs ORK_I4_NOLADDER=1, 10 iters, interleaved ABAB:
| K | ladder ON | NOLADDER |
|---|---|---|
| 512 | 1306 / 1319 µs | 1320 / 1322 µs |
| 1024 | 1738 / 1680 µs | 1712 / 1687 µs |
| 2048 | 2994 / 3012 µs | 3087 / 3059 µs |
Within 1–3% and not consistently signed. Die-neutrality confirmed on the part the claim was made about: no memo entry, no measurable cost.
- §3's four-band
K·Wbtable is aWb=128-shaped slice of a(K, Wb, H)surface. TheWb == 64clause generalises; theK·Wbthresholds do not. - §3a's capacity reading is refuted: equal bytes fall on both sides, the bank split is inert, and
raising
Hrescues the identical tile. The measurements in §3a stand. - §3a's "the only way to use
Wb > 64atK >= 768is K-slicing" is wrong as stated — on.236K=768 Wb=128runs atH>=4, andK=1024 Wb=128atH>=8. The lever isH, which is free. §6a argues largerWbis where the time is; on this part it is reachable at productionKup to 1024, which is worth re-timing. - §4's "the race is one
K" should read "the race is the wholeNBc>=2wall plus theWb=64hole"; theWb=64hole is the special case where the floor exceeds the ceiling atNBc=1. -
ORK_I4_1010remains a trap, untouched here.
15. ★ E1 — the stride-2 zero lines ARE read, and the batch declares FOUR physical rows per logical row
Board .236, uname -r = 6.1.115-vendor-rk35xx-fencefix, DMC performance @ 2112 MHz, cpu4/cpu6
@ 2.4 GHz. Shape K=512 N=64 M=8 H=4 Wb=64 nc=1 → NG=2, NC=1, two programs. Instrumentation is
RE-only and lives in an isolated clone (board:~/work/e1, git archive 494b30c): four env-gated
hooks in src/npu/i4/chain.c — ORK_I4_ODDFILL (fill the odd lines: a byte pattern, c=copy the
even line, s=swap even→odd, z=zero both), ORK_I4_ONLYLINE=<p> (zero the cube, fill exactly one
physical line), ORK_I4_RAWMARK (paint the whole output surface before the submit, so untouched
is distinguishable from written) and ORK_I4_RAWDUMP (dump the raw int16 surface post-completion;
the de-tile only reads it). Not committed.
The surface per program is 4·H·Wmax·64 = 1024 int16 = sixteen 64-int16 slots. bch_db_cells_off
de-tiles logical row j from slot 4j.
| arm | odd lines | de-tiled output (slots 4j) |
whole-output checksum |
|---|---|---|---|
| A |
0x00 (shipped) |
reference | 234bda9b51e0b133 |
| B | 0x33 |
bit-identical | 234bda9b51e0b133 |
| C | 0x55 |
bit-identical | 234bda9b51e0b133 |
| D | copy of the even line | bit-identical | 234bda9b51e0b133 |
Slots 4j+1 … 4j+3 change in every arm, and the change is exactly linear in the odd nibble: on
every element inspected, (C − B) / (B − A) = 2/3 to the integer (nibble 5 vs nibble 3, baseline 0).
ORK_I4_ONLYLINE isolation (cube zeroed, one line filled with 0x11, so every activation element
is 1 and each output is a column sum of B over whatever K-window that row reads):
even line 2j -> slots { 4j, 4j+1, 4j+2, 4j+3 } first values -138 -156 -148 -178
odd line 2j+1 -> slots { 4j+1, 4j+2, 4j+3 } first values +18 +10 +40
and for each spurious slot the two contributions sum to the correct full row:
-156+18 = -148+10 = -178+40 = -138 = slot 4j. An all-zero cube gives an all-zero surface, so none
of this is stale residue.
-
Model 1 (the zeros feed the same accumulator, half the MACs multiply zero) — REFUTED. Four
different odd-line contents leave the de-tiled rows bit-identical. Slot
4jis a pure function of even line2j. - Model 3 (fetched-and-discarded, or not fetched) — REFUTED. The odd lines are read: they move three slots per logical row, linearly. No DMC-bandwidth arm is needed; the output surface settles it.
-
Model 2 (
2Hindependent rows, read them all andHdoubles) — REFUTED. With the even line duplicated into the odd line (arm D), an exhaustive 1-int16-granularity scan found no second copy of any de-tiled row anywhere in the surface. The three extra slots are not rows.
What they are. bch_db_worker calls orki_i4_synth(rc, 2*Hg, …) and orki_i4_synth_full sets
mc_phys = 2*mc, so the hardware is told 4·Hg physical rows for Hg logical ones. It computes
all four as full-K reductions whose activation start offsets step by a sub-row amount inside each
2-line (2·K/2-byte) window: slot 4j starts at the head of the even line and is correct; slots
4j+1…3 start part-way into it and take their tail from the odd line — which is exactly why their
even-part and odd-part sum to the full column sum. Three of every four computed rows are discarded.
So the activation cube costs K bytes per useful row where a dense int4 row is K/2 — the zero lines
are one factor of 2 — and the declared row count is 4H for H useful rows. Hypothesis, not yet
proven: the bank budget is charged per DECLARED row, which would be the unexplained second factor of 2
in H_max = DBNK·16384/K against the dense-row prediction DBNK·32768/K.
ORK_I4_DENSE=1 (rows at stride K/2, orki_i4_synth(rc, Hg, …), de-tile stride 2 instead of 4) is
not bit-exact at {512,64,8,H4}, {512,256,32,H4}, {1024,256,32,H8}, {2048,512,64,H8}.
Note the trap: at K=2048 it was also the fastest arm (350 µs median vs the anchor's 486) while
being wrong. Removing the spurious rows needs the activation row-stride register
(RK_CNA_DMA_CON1/0x107c, currently K/16), not a host-side repack.
16. E2b — declaring the whole cube in CNA_CBUF_CON1 does NOT move the H ceiling; it breaks correctness
int8 declares the whole cube (DATA_ENTRIES = (K/64)·mc, i8/regcmd.c); int4 declares one row's worth
(ceil(K/128), no mc factor, i4/run.c). Arm: CBUF_CON1 = ceil(K/128)·mc_phys (ORK_I4_CON1MC=1),
K=1024 N=256 M=64, 4 reps per H.
H |
baseline distinct checksums |
×mc_phys distinct checksums |
|---|---|---|
| 8 | dd1a… |
3613…, ab55…
|
| 14 | dd1a… |
3af1… |
| 16 | dd1a… |
b6ea… |
| 17 | 2 distinct (over the ceiling) |
de3a…, efb9…
|
| 18, 20, 24, 32 | 3–4 distinct each | one each, all different per H
|
Baseline is stable and identical for H = 8, 14, 16 and goes unstable at H = 17 — i.e. it reproduces
the known ceiling of 16. With ×mc_phys every H produces a different checksum, including H well
below the ceiling: the register does not enlarge the window, it corrupts the reduction. The ceiling
did not move. Not worth another arm.
Method: tools/re/i4_wb_sweep.c (new) — one pack, one context, ORK_I4_H/ORK_I4_WB re-read per call
so a single process walks H upward (never bisected) for each Wb. Every cell graded against the
same-process Wb=64, H=2 anchor (which agreed 3/3 everywhere) on full-random data, ORK_I4_NOLADDER=1,
behind npu_guard.sh --. Both gates: bit-exact and median under 2× the Wb=64 best at that K.
Phase A 2 reps contiguous; phase B 6 reps, ascending and descending.
N=1024, M=64, nc=3.
K (hcap) |
Wb=128 |
Wb=192 |
Wb=256 |
Wb=320 |
|---|---|---|---|---|
| 512 (32) | 2 | 2 | 4 | 3 |
| 768 (22) | 5 | 8* | 8* | 8* |
| 1024 (16) | 8 | 9 | 5 | 7 |
* H=6,7 not sampled at K=768; the floor there is ≤8 and >5.
The floor is non-monotone in Wb and in K. At K=1024, Wb=256 clears at H=5 while Wb=128
needs 8 and Wb=320 needs 7. At fixed Wb=256 the floor is 4 → 8 → 5 as K goes 512 → 768 → 1024.
Neither "rises with K" nor "rises with Wb" survives. There is no law here, only a surface.
Same cell, 6 reps each, ascending vs descending H:
| cell | asc | desc |
|---|---|---|
K=1024 Wb=320 H=5 |
6/6 @ 436 µs | 1/6 @ 16.1 s |
K=1024 Wb=192 H=7 |
2/6 | 6/6 @ 436 µs |
K=768 Wb=256 H=5 |
0/6 | 5/6 |
K=1024 Wb=128 H=6 |
2/6 | 5/6 |
A single-order sweep of the band is not reproducible. This is stronger than the §11 note that 3 reps
cannot tell 1/6 from 0/n: here 6 reps in one order give 6/6 and 6 in the other give 1/6.
Cells that match the anchor bit-exactly while the median is 2–6 orders out:
| cell | pass | median |
|---|---|---|
K=768 Wb=192 H=6 / 7 / 8 (2 reps) |
2/2 each | 4.5 s / 6.6 s / 2.1 s |
K=1024 Wb=192 H=8 (6, desc) |
6/6 | 109 ms |
K=768 Wb=128 H=4 (6, desc) |
6/6 | 218 ms |
K=512 Wb=256 H=4 (2 reps) |
2/2 | 105 ms |
K |
Wb=64 |
128 |
192 |
256 |
320 |
shipped | best | gain |
|---|---|---|---|---|---|---|---|---|
512 (H=31) |
1622/1658 | 1309/1353 | 1164/1217 | 1129/1171 | 1100/1143 |
Wb=256 H=32 1268/1307 |
Wb=320 H=31 |
1.15× |
768 (H=21) |
1823/1818 | 1452/1476 | 1272/1282 | 1214/1214 | 1170/1184 |
Wb=128 H=22 1469/1504 |
Wb=320 H=21 |
1.26× |
1024 (H=15) |
2074/2096 | 1567/1540 | 1390/1376 | 1325/1306 | 1284/1263 |
Wb=128 H=16 1597/1592 |
Wb=320 H=15 |
1.26× |
At this shape and H ≥ 9, all 90 (K, Wb, H) cells are 6/6 in both orders with no inflation. The
Wb gradient is monotone in time and still improving at 320 (the last 256→320 step is worth 3%); 320
was the widest tile sampled.
Side finding, separate lever: H = orki_i4_hcap(K) is a LOCAL WORST wherever it exactly divides
M (64/32, 64/16, 128/32, 128/16 here). H = hcap−1 is 5–12% faster at every Wb and every K
tested — e.g. K=512 Wb=320 N=2048 M=128: H=32 1277 µs, H=31 1100 µs. Measured, unexplained,
and M-dependent, so it is not part of the rule below.
Wb = (K <= 1024 && H >= 12) ? min(320, N) : ((131072 / K) & ~63)
-
Margin. Worst conservative floor measured is 9 (
K=1024, Wb=192); 12 is 1.33× over it.orki_i4_hcapis 32 / 22 / 16 atK= 512 / 768 / 1024, so the production path always takes the wide tile and never sits in the erratic band. -
Partial tiles are fine.
Wb=320divides neitherN=1024(3×320 + 64) norN=2048(6×320 + 128); both were bit-exact at every passing cell, soWmax = Wb/64and theWc = N − n0clamp hold. -
Safety net unchanged. Ladder rung 1 is already
Wb=64and rung 2 addsH=4; a part that refuses the wide tile falls back bit-exact and memoizes on(K, N). -
CAVEAT — one die. Every number here is
.236. §12 records that.235refuses geometries.236accepts. On a part that refusesWb=320, this rule converts a 3.6 ms call into a ~15.9 ms first-call learn per(K,N). Re-measure 17.1 and 17.4 on.235before landing. Not landed for that reason.
- §11's
H_race"RISES with K" is wrong as a law. Measured at fixedWb=256the floor goes 4 → 8 → 5 acrossK= 512 → 768 → 1024, and at fixedK=1024it goes 8 → 9 → 5 → 7 acrossWb= 128 → 192 → 256 → 320. It is a surface; sample it, do not extrapolate it. - §11's
Wb=128floors (K=768 → 4,K=1024 → 8) are confirmed approximately but are optimistic atK=768: at 6 reps in both ordersH=4is 5/6 ascending and 218 ms descending, so the conservative floor is 5. - §6's "geometry tuning beyond the ladder's answer: ≤5%, not shipped" is superseded for
K ≤ 1024: at a production shape theWblever alone is 1.15–1.26× (§17.4), becauseWb=64— the rung the memo pins a refusing shape to — is 1.5–1.6× off the best, not 5%. - The E1 result (§15) retires the "are the odd lines read?" question that §§1–14 left open, and
replaces the stride-2 narrative: the cube is
2Hlines but the hardware is told4Hrows.
Board 10.3.0.236, uname -r = 6.1.115-vendor-rk35xx-fencefix, DMC + A76 governors performance
(A76 at 2 400 000 kHz). Clone ~/work/e2 (instrumented) and ~/work/e2p (the minimal patch), both
from d9c5d15. Tooling: tools/re/i4_e2_probe.c grades every iteration against an absolute CPU
int4×int4 reference (not against iteration 0, which cannot see a consistently-wrong geometry), and
tools/re/i4_e2_match.c reads an ORK_I4_RAWDUMP surface and, for the one activation line that
ORK_I4_ONLYLINE filled with all-ones, searches every contiguous [lo,hi) for an exact 64-column
match — which converts the slot map into a statement about addresses.
With mc_phys declared rows, declared row R reduces over K contiguous activation elements
starting at
byte_offset(R) = ( (R >> 2) * LINE_STRIDE + (R & 3) ) * 16
\_______/ \_____/
group of 4 one 16-byte atom
where LINE_STRIDE is 0x107c CNA_DMA_CON1. Two independent, separately-measured facts:
-
0x107cstrides GROUPS OF FOUR, in units of 16 bytes. Sweeping it atK=512(one int4 row = 256 B = 16 atoms) moves the base of group 1 exactlyvalue × 16bytes:0x107cgroup-1 base = physical line 8 128 B half a row 16 256 B line 1 32 (shipped, K/16)512 B line 2 64 1024 B line 4 This is why the shipped pack must interleave a zero line: at
K/16the group stride isKbytes = two int4 rows. The earlier reading of0x107cas "the batch activation-cube-size reg, K/16 restores the 4-ROW batch" (Exp-2026-07-07) was describing this side effect, not the register. -
The step WITHIN a group is a hardwired 16-byte atom. Measured identical at
K=512andK=1024, and invariant under every register tried:0x107cover a 32× range,0x1044CBUF_CON1,0x1010CONV_CON2,0x102cDATA_SIZE3,0x104c,0x1078DMA_CON0DATA_BURST_LEN(0xf → 0x7 → 0x3 all give 4; 0x1 and 0x0 fracture the reduction incoherently), DPU0x4030 DATA_CUBE_WIDTH, DPU0x4024 DST_SURF_STRIDE, andmc_physitself.
So at K=512 the four windows in a group are bytes [0,256) [16,272) [32,288) [48,304) — they
overlap by 240 of 256 bytes. One of the four is a logical row; the other three are
sub-row-shifted duplicates of it, and no packing can make them anything else.
-
0x1020 CNA_DATA_SIZE0WIDTH[26:16]is the only thing that changed the within-group step: atK=512,W=4gives 64 B instead of 16 B (slots read[0,512) [0,384) [0,256) [0,128)of the filled line). It does not change the group size, and atW≥8the reduction stops being contiguous (W=8→[0,256) [0,256) [0,128) [0,128);W=16→ all four[0,128)), so it cannot be pushed to theK/32atoms a dense row would need. AtK=128,W=4gives 16 B again, so the step is not simplyWatoms either. -
0x1028 CNA_DATA_SIZE_BATCH = 4relocates the four rows of a group to output slots0, 4, 8, 12with the same four activation windows. It is output placement, not computation.
Pack one logical row per physical line (K/2 B apart) and set 0x107c = K/32. The two must move
together: a dense pack at the shipped K/16 reads only the EVEN logical rows and is silently
wrong — which is exactly what E1's ORK_I4_DENSE=1 arm measured (§15's "a host-side dense repack
alone does NOT work"), now explained rather than observed. mc_phys is unchanged at 4·Hg, the
output surface is unchanged, and the de-tile still reads every 4th slot.
6 reps, dm=1 (full random), ascending and descending, median µs, every cell 6/6 bit-exact
against the CPU reference:
| K | N | M | H | shipped | dense | ratio |
|---|---|---|---|---|---|---|
| 512 | 2048 | 64 | 32 | 798 | 790 | 1.01× |
| 1024 | 2048 | 128 | 16 | 1729 | 1582 | 1.09× |
| 2048 | 2048 | 128 | 8 | 2983 | 2607 | 1.14× |
| 2048 | 2048 | 100 | 8 | 2380 | 2092 | 1.14× |
| 2048 | 512 | 256 | 8 | 1689 | 1428 | 1.18× |
| 2048 | 2048 | 128 | 16 (DBNK=2) |
2907 | 2490 | 1.17× |
| 4096 | 1024 | 64 | 4 | 1212 | 1201 | 1.01× |
ORK_I4_DENSE=0 reproduces the shipped numbers exactly, in both arm positions. The gain is the
halved activation DMA — it tracks how often the activation is re-read, i.e. the N-tile count, and
is ~nil at K=4096 where the weight stream dominates.
make test ALL TESTS PASSED on .236 with the pack as the default, in both clones.
Minimal patch (57 changed lines across i4/chain.c, i4/run.c, i4/i4.h, tools/size_budget.txt)
at board:~/work/e2_dense.patch; its tests/sbc_attest.txt CORE_SHA is
a31c7df3c5f060380b2e095b8df3f30ccd83394f7b76d83f93d850967f67ad6d. Guarded on K % 32 == 0 (the
group stride K/32 must be exact; every K in orki_i4_hcap's table qualifies) and opt-out via
ORK_I4_DENSE=0. The thread-local that tells orki_i4_synth which packing it is emitting for is
part of the template-cache key — without that a cached template crosses between the two strides.
CAVEAT — one die. Every number is .236; re-measure on .235 before landing.
The dense pack halves the supplied activation bytes (H·K/2 instead of 2H·K/2) and nearly halves
the DRAM span, while leaving mc_phys = 4·Hg untouched. Scanning H upward (never bisecting) at
K=2048, N=2048, M=128:
| arm | DBNK | H=8 | H=9 | H=10 | H=12 | H=16 | H=17 |
|---|---|---|---|---|---|---|---|
| shipped | 1 | exact | — | — | — | — | — |
| dense | 1 | exact | WRONG | WRONG | — | — | — |
| shipped | 2 | exact | — | — | exact | exact | WRONG |
| dense | 2 | exact | — | — | exact | exact | WRONG |
The two arms ceil at exactly the same H at both bank splits: 8 at DBNK=1, 16 at DBNK=2,
i.e. DBNK · 16384 / K unchanged.
- REFUTED (measured): the ceiling is a budget on the activation bytes actually supplied, or on the DRAM span the DMA walks. Both were roughly halved and nothing moved.
-
HOLDS (consistent with everything measured, not yet causally isolated): the budget is charged
on the declared geometry —
mc_phys · (K/2) ≤ DBNK · 32768, which withmc_phys = 4His exactlyH_max = DBNK · 16384 / K. This is the long-unexplained second factor of 2 in that formula: it is the 4× declared-row padding, charged in full. -
UNVERIFIED: isolating it causally needs
mc_physINFLATED (2× → H_maxshould halve) with themccoutput andmafactivation allocations enlarged to match. Not run — the allocations are sized from4*H*Wmaxand2*Hat six sites, and getting one wrong is a DMA overrun on the measurement board. Specified here as the next step, not attempted.
The padding disappears exactly when one dense int4 row is one 16-byte atom, i.e. K = 32. At
K=32, N=64, M=8, H=4 with a dense pack, mc_phys = Hg, 0x107c unused (one group) and the
de-tile at stride 1, the line → slot map is 1:1 (line 0→slot 0, 1→1, 2→2, 3→3) and the result is
6/6 bit-exact. At K=128 the same arm gives the usual four 16-byte-apart overlapping windows: a
dense row is 64 B there, four atoms wide.
So the useful fraction is min(4, K/32)⁻¹: no padding at K=32, 2× at K=64, and 4× at every
K ≥ 128 — i.e. at every production K. The 4× is not removable, because the within-group
activation step is a hardwired 16-byte atom and a dense int4 row is K/2 ≥ 64 bytes, so three of
every four declared rows are reductions over a window that overlaps the real row by
K/2 − 16 bytes. No register among the eleven swept changes that step without destroying the
fetch. The remaining int4 dispatch lever is therefore Wb (§17), already measured at 1.15–1.26×,
plus the dense pack's 1.09–1.20× above, which is bandwidth and host work rather than rows.
- E1's "sub-row activation start stride inside each 2-line window" is right in substance. The exact
law is §19.1: atom-indexed, groups of four, the group stride being
0x107c × 16bytes. - E1's measured slot values reproduce exactly (
−138 −156 −148 −178/+18 +10 +40at column 0, seed 777,K=512 N=64), and the k-ranges behind them are[0,512) [0,480) [0,448) [0,416)from the even line and[480,512) [448,512) [416,512)from the odd one. - "
ORK_I4_DENSE=1is not bit-exact" was a missing paired register, not a property of dense packing: it is bit-exact with0x107c = K/32(§19.3). The K=2048 arm that was "the fastest and wrong" is now the fastest and right.
Board 10.3.0.236, uname -r = 6.1.115-vendor-rk35xx-fencefix, DMC performance @ 2112 MHz,
A76 performance. 6 reps per cell, every sweep run ASCENDING and DESCENDING (per
ab-arm-position-bias), full-random data (dm=1), graded against the ABSOLUTE CPU reference in
tools/re/i4_e2_probe.c — i4_storm_probe grades against its own iteration 0 and every arm here
CHANGES the geometry. Every number below is the mean of the ascending and descending medians, in
µs; the first iteration of each cell is a ~106 ms cold warm and is excluded by the median.
N=2048 throughout. Unless stated, ORK_I4_NOLADDER=1 with ORK_I4_H / ORK_I4_WB pinned.
H = orki_i4_hcap(K) — the shipped value — is a local worst. At K=512 Wb=320 M=128,
H=32 measures 1326 µs and H=31 measures 1147, bit-identical, although H=31 raises
the program count (NG 4 → 5, NT per core 12 → 15). The effect had been seen only at
(M, H) ∈ {(64,32), (64,16), (128,32), (128,16)}, where H both divides M and is a power of
two, so two hypotheses were indistinguishable: A H | M, B a power-of-two H (or a
power-of-two group stride H·K). Both are refuted below. A third, D "H == hcap is special",
is refuted too.
M=130, K=512, Wb=320. Every H in 26..32 gives NG=5, so the program count, the core split
and the total row count are all identical; only the split between the full groups and the tail
changes. r = M−(NG−1)·H is the final group's row count.
H |
26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|
r |
26 | 22 | 18 | 14 | 10 | 6 | 2 |
| µs | 1280 | 1305 | 1284 | 1252 | 1215 | 1193 | 1173 |
H=26 divides 130 and is the SLOWEST; H=32 is a power of two, is hcap, and is the FASTEST.
A, B and D all predict the opposite. The ordering tracks r.
K=512, Wb=320, M=128. (M=128 is where A and B are perfectly confounded: 128 = 2⁷, so
every divisor of M is a power of two.)
H |
16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
NG |
8 | 8 | 8 | 7 | 7 | 7 | 6 | 6 | 6 | 6 | 5 | 5 | 5 | 5 | 5 | 5 | 4 |
r |
16 | 9 | 2 | 14 | 8 | 2 | 18 | 13 | 8 | 3 | 24 | 20 | 16 | 12 | 8 | 4 | 32 |
| µs | 1285 | 1184 | 1152 | 1255 | 1189 | 1189 | 1291 | 1233 | 1191 | 1172 | 1254 | 1319 | 1292 | 1225 | 1143 | 1138 | 1306 |
Within every NG band the time falls as H rises to the top of the band, then jumps when the band
changes. NG itself barely matters: at r ≈ 2–4 the four bands give 1152 / 1189 / 1172 / 1138,
and at r ≈ 16–18 they give 1285 / 1291 / 1292. A one-variable fit T ≈ 1140 + 8·r reproduces
both this table and §21.2 to within ~5% everywhere except r ≥ 24, where it saturates.
hcap(768) = 22, not a power of two. M=132, Wb=320.
H |
22 | 21 | 20 | 19 | 18 | (M=133, H=22) |
|---|---|---|---|---|---|---|
NG |
6 | 7 | 7 | 7 | 8 | 7 |
r |
22 | 6 | 12 | 18 | 6 | 1 |
| µs | 1345 | 1237 | 1286 | 1353 | 1282 | 1217 |
H=19 does not divide 132 and is the joint-slowest; H=18 does not divide it and is fast.
So it is not A. hcap=22 is not a power of two and is slow; H=32 at K=512 is a power of two
and is fast (§21.2). So it is not B. Slope here: (1345−1217)/21 = 6.1 µs per row of r.
K=512, Wb=320, H=32 fixed. Work is proportional to M; r = M−128 for M ≥ 129.
M |
128 | 129 | 132 | 136 | 144 | 152 | 160 | 160 (H=31) |
|---|---|---|---|---|---|---|---|---|
NG |
4 | 5 | 5 | 5 | 5 | 5 | 5 | 6 |
r |
32 | 1 | 4 | 8 | 16 | 24 | 32 | 5 |
| µs | 1400 | 1157 | 1205 | 1246 | 1363 | 1502 | 1592 | 1384 |
M=128 → 129 is +0.8% work and −21% time. And at a fixed M=160, dropping H 32 → 31 (more
programs, same rows) is −13%. Fitting T = c + β·M + γ·r over this table gives β ≈ 6.3 µs per
row and γ ≈ 7.7 µs per row of the tail group — a tail row costs roughly double.
Conclusion: the variable is r, the row count of the FINAL M-group. H | M is simply its
maximal case (r == H), which is why H = hcap looked like the culprit whenever hcap divided
M.
ORK_I4_BLOCKING=1 (bare blocking submit, prepolled=1, no SENT16 seed and no
drain-verify poll — it miscomputes, so this is a timing-only arm and the verdicts are FLAKY):
| cell | r |
NONBLOCK (shipped) | BLOCKING |
|---|---|---|---|
M=128 H=32 |
32 | 1400 | 803 |
M=129 H=32 |
1 | 1157 | 827 |
M=160 H=32 |
32 | 1592 | 973 |
M=160 H=31 |
5 | 1384 | 994 |
The entire r penalty disappears, and the work ordering is restored (M=128 now faster than
M=129; H=32 now faster than H=31 at M=160). So the cost is host-side, in the completion
path: ork_dyn_end gates on the LAST program of the chain and bch_db_cells scans that program's
r·NBc·64 output cells, returning at the first still-sentinel cell, so the scan is re-walked every
poll pass for the whole execution window of that program. A short tail group is a short scan.
This predicts the penalty scales with the N-tile width, since NBc = Wc/64.
K=512, M=128, dense pack on (the §19 default).
Wb |
Wmax |
H=32 |
H=31 |
gain |
|---|---|---|---|---|
| 64 | 1 | 1661 | 1679 | 0.99× (a small loss) |
| 128 | 2 | 1460 | 1357 | 1.076× |
| 256 | 4 | 1316 | 1186 | 1.110× |
| 320 | 5 | 1326 | 1147 | 1.156× |
At the currently shipped Wb for K=768 and K=1024 (both 128) the lever is a wash —
K=1024 M=128: H=16 1615, 15 1578, 14 1582, 13 1663; K=768 M=132: H=22 1554, 21 1538,
20 1610, 18 1543. So the Wb lever of §17 and this one are NOT independent: this one barely
exists without it. Combined at K=512 M=128: Wb 64→320 alone is 1.253×, H 32→31 alone (at
Wb=64) is 0.989×, and the pair is 1.449× — super-additive, because widening the tile is what
makes the tail scan expensive enough to be worth routing around.
K=512 N=2048 M=128, 2×2×2 over Wb × ORK_I4_DENSE × H:
Wb |
DENSE |
H=32 |
H=31 |
H lever |
|---|---|---|---|---|
| 256 | 0 | 1356 | 1208 | 1.122× |
| 256 | 1 | 1307 | 1210 | 1.080× |
| 320 | 0 | 1330 | 1164 | 1.143× |
| 320 | 1 | 1336 | 1140 | 1.172× |
The H lever is 1.08–1.17× with the dense pack and 1.12–1.14× without it: independent, and it
stacks. (The dense pack's own effect at this particular shape is small, ≤ 3.6%; its 1.09–1.20×
was measured elsewhere.)
K=2048 Wb=64 M=128, where Wb is pinned at 64 by the bank rule: H=8 2621, H=7 2704,
H=6 2752, and M=129 H=8 2602. The shipped H is already optimal there — r=8 costs ~48 µs
on a 2600 µs base while NG 16 → 19 adds ~33 programs to the critical core. Lowering H at
Wb=64 is a 3–5% LOSS; any rule must be gated.
orki_i4_bchain_try, after Wb/NC/nc are resolved (both are needed: the trade is tail-poll
rows against programs-per-critical-core). A 4-wide search over H ∈ [hcap−3, hcap] minimising
cost(h) = (Wmax + Wmax/4)·r(h) + 3·⌈M/h⌉·⌈NC/nc⌉
gated on Wb ≥ 256, H ≥ 12, M > H, and h ≥ 8. ORK_I4_NOTAIL=1 reverts. The model picks the
measured optimum at all four sweeps above (K=512 M=128 → 31, M=130 → 32, K=768 M=132 →
21, K=1024 M=128 → 14).
Production A/B — no ORK_I4_* geometry overrides, the real Wb=(131072/K)&~63 = 256,
K=512 N=2048, both arm orders, checksums identical to the off arm at every point:
M |
tail-OFF (H=32) |
tail-ON (H=31) |
gain |
|---|---|---|---|
| 128 | 1340 | 1179 | 1.137× |
| 192 | 1883 | 1751 | 1.075× |
| 256 | 2528 | 2343 | 1.079× |
Controls, both arms byte-identical in geometry and timing: K=2048 M=128 keeps H=8 (2992 vs
3022, Wb=64 → gate off); K=512 M=130 and M=100 keep H=32 (the cost model declines to
move). make test ALL PASSED on .236; attest refreshed.
Reach today is K ≤ 512, since Wb ≥ 256 requires it. If §17's Wb widening lands for
K ≤ 1024, the gate opens for K=768 and K=1024 too, at the 1.08–1.17× measured in §21.7.
-
The 4× declared-row padding (§20) is untouched by this.
ris a count of LOGICAL rows; the poll walks the written cell set, which is already padding-free. -
The real fix is probably not geometric. The penalty is a poll that re-scans a whole program's
surface; a poll gated on a small witness set per program would delete it rather than route around
it, and would also recover the ~40% that §21.6's BLOCKING arm shows is being spent on the
drain-verify (803 vs 1400 µs at
M=128). That arm miscomputes, so the 40% is not attainable as it stands — but it bounds what a cheaper CORRECT gate is worth, and it is far more than 1.14×. -
γwas measured only atK≤ 1024 andN=2048. TheWmaxscaling in the cost model is a two-point fit (§21.7) and is not claimed beyondWb≤ 320. -
Nothing here was cross-validated on
.235, whose DRAM clock changes the dispatch envelope (§§6, 17). The gate is conservative in the right direction — it only ever LOWERSH, which is the direction the ladder already treats as safe — but the cost model's constants are.236's.
Board 10.3.0.235, uname -r = 6.1.115-vendor-rk35xx-fix, DMC governor performance at a verified
2 400 000 000 Hz, A76 performance. Fresh isolated clone of c2eb677 at ~/work/ork235-wb
(baseline) and ~/work/ork235-wbfix (patched); every NPU command behind
sudo tools/util/npu_guard.sh -- timeout -s TERM ….
New tool: tools/re/i4_ladder_probe.c. It grades against an ABSOLUTE CPU int4×int4
reference (§19's rule — i4_storm_probe grades against its own iteration 0 and i4_wb_sweep
against a Wb=64 H=2 anchor, and both anchors are themselves geometries, which a ladder A/B changes
by construction). Two modes: a named (Wb,H) cell list walked in a given order inside one process,
and a LADDER mode with no geometry override that prints per-rep times, so the one-time learn is
separable from the steady state. Full-random data, 6 reps, both arm orders, each arm its own process.
§17.5 proposed Wb = (K<=1024 && H>=12) ? min(320,N) : ((131072/K)&~63), worth 1.15–1.26× on
.236, and blocked it on "re-measure on .235 first". Measured here, the rule alone is −25% at
K=768 (1532 → 1917 µs steady state).
It is not the rule and it is not the learn. The learn costs ~1.3 s, not 15.9 s — the (K,N)
memo and ladder_probe (§13) both work. The cause is that ladder rung 1 jumped straight to
Wb=64, which is BELOW today's default width. So a part that refuses Wb=320 at K=768 forfeits
the Wb=128 it does take and lands two rungs down, permanently for the process.
A tighter H threshold is not an available fix: on .235 the refusals happen at H = hcap, already
the maximum. The binding axis is Wb.
Cells at N=2048 M=128, ORK_I4_NOLADDER=1 ORK_I4_NOTAIL=1, absolute reference.
K (hcap) |
Wb=128 |
192 |
256 |
320 |
reps |
|---|---|---|---|---|---|
| 512 (32) | 6/6 | 2/2 | 2/2 | 2/2 | 6 / 2 / 2 / 2 |
| 768 (22) | 6/6 | 0/2 | 0/2 | 0/2 | 6 / 2 / 2 / 2 |
| 1024 (16) | 1/6 asc, 0/6 desc | — | — | — | 6 |
A pass runs in 1.3–1.6 ms; every refusal is ~15.6 s (accepted submit, output never lands, six
recover rounds). At K=512 the three wide widths are within 2% of each other at H=32
(min 1310 / 1334 / 1304 µs for 192 / 256 / 320).
.235 takes the wide tile at K=512 and refuses every width above the shipped floor at K=768.
The contrast with §17.4, where .236 is 6/6 at all four widths and all three K, is the whole of
the problem: the rule is a per-part guess, and only the ladder can make it safe.
rung 0: Wb = orki_i4_wbdef(K,N) min(320,N) at K<=1024 && hcap>=12, else the floor
rung 1: Wb = (131072/K) & ~63 <-- NEW: today's shipped width
rung 2: Wb = 64
rung 3: Wb = 64, H = 4
orki_i4_wbdef lives in src/npu/i4/run.c and is orki_i4_bchain_try's default Wb;
ORK_I4_NOWIDE=1 pins it to the floor for an A/B. The "a rung only EXISTS where it is a DIFFERENT
geometry from the live rung above it" rule (§13) is generalised over the four, so a part where
wbdef already is the floor, or where Wb is already 64, pays nothing for the extra rungs.
Both new steps only re-tile M/N — same tiles, same kernel — so every rung is bit-exact.
N=2048 M=128, 6 reps, medians, asc / desc arm order, each arm its own process.
A = shipped c2eb677; B = patched build with ORK_I4_NOWIDE=1 (ladder fix only); C = patched build
(ladder fix + Wb rule). Every arm, every K: 6/6 BIT-EXACT against the CPU reference.
K |
A shipped | B ladder fix only | C fix + Wb rule |
C vs A (2-order mean) | A/A spread |
|---|---|---|---|---|---|
| 512 | 1188 / 1213 | 1195 / 1147 | 1177 / 1097 | −5.3% | 5.6% (1154–1219) |
| 768 | 1559 / 1591 | 1572 / 1556 | 1573 / 1585 | +0.3% | — |
| 1024 | 2203 / 2163 | 2205 / 2215 | 2226 / 2212 | +1.6% | 1.3% (2200–2229) |
| 2048 | 3512 / 3579 | 3587 / 3548 | 3529 / 3547 | −0.2% | — |
-
The −25% at
K=768 does not occur.ORK_BCH_DEBUGconfirms the dispatched geometry: rung 0H=21 Wb=320is refused once, then every call runsH=22 Wb=128— today's width. 1573 against 1559, where rung 1's absence would have put it at 1917. -
K=1024's +1.6% is not a geometry effect: A and C dispatch the SAME geometry there (Wb=64H=16, both having been refused above it), and the A/A spread at thatKis 1.3%. -
K=512's −5.3% is NOT established as a gain. It is the oneKwhere the wide tile is taken, and C is at or below every A observation — but the A/A spread is 5.6%, which swallows it. Call it "≤ a few percent, direction favourable". -
The rule's cost on this part is the LEARN, not the steady state.
K=768 now pays 1.32 s once per(K,N)per process where it previously paid none;K=1024 pays 2.54 s instead of 1.33 s (one extra refused rung). Bounded by the 64-entry memo and by distinct(K,N)per model.
Verdict: ship it. On .235 the combination is non-regressive at all four production K within
the board's own A/A spread, bit-exact everywhere, and .236 keeps its 1.15–1.26×. The one real cost
is ~1.3 s of extra first-call learn at K=768 and K=1024 on a part that refuses the wide tile.
This predates the rule entirely and is the more important finding. Same method
(ORK_I4_NOLADDER=1 ORK_I4_NOTAIL=1, absolute reference, 6 reps, both orders, N=2048 M=128):
| cell | asc | desc | the rung it falls to |
|---|---|---|---|
K=1024 Wb=128 H=16 — SHIPPED
|
1/6 | 0/6 |
Wb=64 H=16, 6/6, 2161 / 2233 µs |
K=2048 Wb=64 H=8 — SHIPPED
|
0/6 | 0/6 |
Wb=64 H=4, 6/6, 3629 / 3499 µs |
Misses are ~15.6 s each. So every K=1024 and K=2048 int4 BCHAIN call on this part has been
running one rung down, silently, since the ladder landed — paying a ~1.33 s learn per (K,N) per
process and then the narrower geometry forever. The ladder has been covering for it, which is
exactly what it is for, but nothing said so.
Order changes the pass rate again (§17.2): K=1024 is 1/6 ascending and 0/6 descending. A
single-order, single-rep check would have called it either dead or alive.
-
K=2048's entry (8) is not this part's ceiling. 4 is — 6/6 in both orders atWb=64, where 8 is 0/6 in both. This agrees with §11's H-scan and with the row already inOPS_REGISTRY. -
K=1024's entry (16) is NOT at fault.H=16 is 6/6 atWb=64. The failing axis there isWb, as §10 found — the table is being blamed for a width problem. -
Neither is a table error to patch. Both are per-part DISPATCH races (§10): accepted submit,
output never lands, fraction rising as cores are removed. A second
hcaptable keyed on the part would encode a race as a capacity law. The remedy is the ladder, and the lesson is thatorki_i4_hcapmust be read as "the.236measurement", never as hardware.
One detail worth keeping: the single K=1024 Wb=128 rep that DID land ran in 1663 µs against
2161 µs for the rung it falls to. The shipped width is the FAST geometry at that K — 1.30× — and
this part simply cannot dispatch it reliably. There is real throughput sitting behind whatever that
race is.
§21.9 shipped TAIL-GROUP BALANCE gated on Wb>=256, because at Wb=128 .236 measured null. On
.235 it is a gain. Cells at Wb=128, 6 reps, both orders, absolute reference, all 6/6
bit-exact (medians asc / desc):
K (hcap) |
H=hcap |
H=hcap−1 |
H=hcap−2 |
H=hcap−3 |
gain at hcap−1 |
|---|---|---|---|---|---|
| 768 (22) | 1579 / 1549 | 1528 / 1516 | 1554 / 1525 | 1564 / 1596 | 2.7% |
| 512 (32) | 1455 / 1486 | 1400 / 1444 | 1405 / 1459 | 1418 / 1468 | 3.3% |
End-to-end with the gate lowered to Wb>=128 (a one-character change in orki_i4_bchain_try),
K=768 is the only production K where .235's ladder lands on Wb=128 — ORK_BCH_DEBUG
confirms the gate moves H 22 → 21:
| arm | asc | desc | mean |
|---|---|---|---|
gate Wb>=256 (shipped) |
1501 | 1572 | 1536 |
gate Wb>=128
|
1509 | 1479 | 1494 (2.8%) |
Controls unmoved: K=512 (both arms at Wb=320, gate already on — a pure A/A, spread 1.2–1.8%) and
K=1024 / K=2048 (both at Wb=64, gate off in both). The per-order numbers show the position bias
of §17.2 / the ab-arm-position-bias note — the first arm was faster in BOTH runs — which is why
only the two-order mean is quoted, and it agrees with the cell-level 2.7% taken the same way.
NOT CHANGED. The gate stays at Wb>=256 until .236 is asked the same question. It measured
null-not-loss there, and a cross-board constant should not be moved on one board's data — which is
the mistake §§17.5 and 23 are both about.
- "Safety net unchanged. Ladder rung 1 is already
Wb=64" was the error. Rung 1 being below the shipped width is precisely what made the rule unshippable: the fallback was not neutral, it was a two-rung drop. The net is now a net. - "
orki_i4_hcapis 32 / 22 / 16 atK= 512 / 768 / 1024, so the production path always takes the wide tile" holds only on.236. On.235the production path takes the wide tile atK=512 and nowhere else. - The CAVEAT is discharged: §§17.1 and 17.4 are re-measured here, the rule is non-regressive on
.235once rung 1 exists, and it is landed.
Board 10.3.0.235, uname -r = 6.1.115-vendor-rk35xx-fix, DMC governor performance at a
verified 2400 MHz, A76 performance at 2400 MHz (re-read before and after every stage, never
below). Probe: i4_ladder_probe (ABSOLUTE CPU int4×int4 reference; both gates reported —
BIT-EXACT and NOT-TIME-INFLATED). Full-random data. ORK_I4_NOLADDER=1 ORK_I4_NOTAIL=1 for every
cell, so the named (Wb,h) is what runs. 6 reps × both arm orders per cell unless stated. Raw
logs: board:~/ork-outbox/tail235-validation/.
At fixed M/K/N/Wb, a candidate h is fully described by g = ceil(M/h) and
r = M-(g-1)h, and (g,r) is a bijection with h. So the model's own form IS the regression:
T(h) ~ alpha + A*g + B*r
B is the per-tail-row cost the gam*r term prices; A is the per-M-group cost the
3*g*tcrit term prices, so Pp = A/tcrit is the per-program cost. At M=128 with hcap=32
the design is confound-free without any modelling assumption: h=31..26 all give g=5 with
r = 4/8/12/16/20/24 (a pure B axis, zero g confound), and h = 30/24/20/15 all give r=8
at g = 5/6/7/9 (a pure A axis). 13 cells per grid, h>=16.
The gate scans the last program's r × NBc 64-int16 blocks (bch_db_cells_off, mode 3/6), i.e.
r*Wmax blocks, so a per-row cost proportional to Wmax with ~no intercept is the prediction.
| grid | Wmax |
tcrit |
alpha µs |
A µs/group |
Pp = A/tcrit |
B µs/tail-row |
fit rms |
|---|---|---|---|---|---|---|---|
| K=512 Wb=64 | 1 | 11 | 1224.7 | 14.90 | 1.35 | 1.21 | 5.2 |
| K=256 Wb=128 | 2 | 6 | 869.5 | 13.11 | 2.19 | 3.35 | 7.8 |
| K=768 Wb=128 | 2 | 6 | 982.1 | 27.75 | 4.62 | 4.64 | 9.1 |
| K=512 Wb=128 | 2 | 6 | 1097.2 | 0.86 | 0.14 | 2.06 | 73.2 |
| K=512 Wb=192 | 3 | 4 | 805.7 | 17.69 | 4.42 | 5.96 | 5.7 |
| K=512 Wb=256 | 4 | 3 | 804.6 | 8.37 | 2.79 | 7.19 | 8.9 |
| K=256 Wb=256 | 4 | 3 | 751.3 | 8.59 | 2.86 | 7.12 | 6.8 |
| K=512 Wb=320 | 5 | 3 | 819.3 | 4.00 | 1.33 | 7.09 | 16.4 |
| K=256 Wb=512 | 8 | 2 | 815.1 | 1.02 | 0.51 | 13.02 | 22.1 |
(italic = A ill-conditioned, see §26.5.) B/Wmax across the clean rows is 1.21 / 1.68 / 2.32 /
1.99 / 1.78 / 1.78 — B ≈ 1.7*Wmax µs/tail-row — and it is K-INDEPENDENT at fixed Wmax
(3.35 at K=256 against 4.64 at K=768, same Wmax=2), as a host-side cache scan must be.
gam is defined by the model's own units (3 per program on the critical core), so
gam = 3*B/Pp. On the six well-conditioned grids: 2.68 / 4.60 / 3.01 / 4.04 / 7.73 / 7.46 at
Wmax = 1 / 2 / 2 / 3 / 4 / 4. A through-origin least squares gives 1.82*Wmax; the three
ill-conditioned grids land at 43.3, 16.0 and 76.7, i.e. all ABOVE that line, so a linear rule is
the conservative read at wide Wb. Shipped was Wmax+Wmax/4 = 1.25*Wmax. New: gam = 2*Wmax.
M sweep, window h ∈ [hcap-3, hcap], measured argmin against each rule's pick. The gain is
against h = hcap, the geometry the search moves away from.
| grid | M | measured best | gain vs hcap
|
1.25*Wmax picks |
2*Wmax picks |
|---|---|---|---|---|---|
| K=512 Wb=128 | 64 / 96 / 128 / 192 / 256 | h=31 | −13.7 / −11.6 / −8.0 / −4.4 / −3.4 % | h=31 ✓ | h=31 ✓ |
| K=512 Wb=256 | 64 / 96 / 128 / 192 / 256 | h=31 (h=30 ties at M=96) | −31.8 / −21.5 / −18.5 / −10.1 / −8.3 % | h=31 ✓ | h=31 ✓ |
| K=768 Wb=128 | 64 | h=20 | −5.0 % | h=21 ✗ (refused — §26.6) | h=20 ✓ (with the guard) |
| K=768 Wb=128 | 96 | h=22 | 0 (A/A) | h=22 ✓ | h=22 ✓ (with the guard) |
| K=768 Wb=128 | 112 | h=22 | 0 (A/A) | h=22 ✓ | h=22 ✓ |
| K=768 Wb=128 | 128 | h=21 | −3.4 % (72 samples) | h=21 ✓ | h=21 ✓ |
| K=768 Wb=128 | 144 | h=20 | −0.6 % | h=22 ✗ (+0.6 %) | h=20 ✓ |
| K=768 Wb=128 | 160 | h=22 | 0 (A/A) | h=22 ✓ | h=22 ✓ |
| K=768 Wb=128 | 192 | h=21 | −2.6 % | h=21 ✓ | h=21 ✓ |
| K=768 Wb=128 | 256 | h=21 | −1.7 % | h=21 ✓ | h=21 ✓ |
3*Wmax picks identically to 2*Wmax on all 18, so it does not discriminate; the smaller
constant is taken (it is also the one nearer .236's slower-DRAM regime — §26.7).
End-to-end on the SHIPPED path (ladder ON, no geometry override, 8 reps, both arm orders, 20
samples per arm, ON against ORK_I4_NOTAIL=1):
| K | M | ON µs | OFF µs | Δ | note |
|---|---|---|---|---|---|
| 768 | 64 | 662.7 | 711.5 | −6.86 % | H 22→20 |
| 768 | 96 | 943.0 | 949.1 | −0.64 % | A/A — the search does not move |
| 768 | 128 | 1199.1 | 1245.7 | −3.74 % | H 22→21 |
| 768 | 144 | 1345.9 | 1355.4 | −0.70 % | H 22→20 — the gam-only case |
| 768 | 192 | 1747.4 | 1792.6 | −2.52 % | H 22→21 |
| 768 | 256 | 2324.8 | 2385.1 | −2.53 % | H 22→21 |
| 1024 | 64…256 | — | — | −0.63 %…+0.57 % |
A/A — falls to Wb=64, gate cannot fire |
| 2048 | 64…256 | — | — | −0.83 %…+0.70 % | A/A — same |
So the A/A noise band at 20 samples/arm is ±0.65 %. ORK_BCH_DEBUG confirms the shipped path
reaches Wb=128 with H = 21 / 20 / 21 / 21 at M = 128 / 144 / 192 / 256.
An 8-sample end-to-end A/B is NOT enough here. At 8 samples the same two K=768 points read −0.12 % (M=128) and +1.06 % (M=144) — the second an apparent LOSS. At 20 samples they are −3.74 % and −0.70 %, agreeing with the 72-sample cells. Do not read an intermediate block.
A = dT/dg at fixed r is a SMALL DIFFERENCE of opposing terms. An extra M-group adds tcrit
programs (regcmd build + dispatch) but SHRINKS the padded mcc sync extent, which is
512*Wmax*tcrit*g*h bytes and falls as g*h = g(M-r)/(g-1) with g. Weight reuse
(ORK_I4_WREUSE, default on) removes the weight re-stream from the extra group, so the surviving
term is small. At tcrit=2–3 the regression then has almost no g leverage and A collapses
(1.02 µs at Wb=512), which inflates gam = 3B/Pp to 76.7. Those rows are reported, not fitted.
The search minimises r, so it walks straight at r=1. Measured:
| K | Wb | M | h | g | r | result |
|---|---|---|---|---|---|---|
| 768 | 128 | 43 | 21 | 3 | 1 | 0/3 — doorbell miss, ~15.6 s each |
| 768 | 128 | 64 | 21 | 4 | 1 | 0/6 |
| 768 | 128 | 96 | 19 | 6 | 1 | 0/6 |
| 768 | 128 | 44 | 21 | 3 | 2 | 3/3 OK, 520.6 µs |
| 768 | 128 | 65 | 21 | 4 | 2 | 3/3 OK, 730.3 µs |
| 512 | 128 and 256 | 64 | 21 | 4 | 1 | 3/3 OK — so it is NOT a universal law |
| 512 | 256 | 63 | 31 | 3 | 1 | 3/3 OK, 522.7 µs — against h=30 (r=3) at 496.7 µs, so r=1 is not even a win |
r=2 at the SAME h passes where r=1 fails, so the variable is r, not h. The guard is
therefore r>=2 in the candidate loop: conservative (it never excludes a measured-better
geometry), and without it the recalibrated model steers K=768 M=64 and M=96 into a 15.6 s miss.
The hole is NOT created by the recalibration — the shipped Wb>=256 gate already picks r=1 at
K=512 M=63 — and it is still reachable when the search does not fire (natural H=hcap with
M % hcap == 1). That residue belongs to orki_i4_hcap, not to the cost model.
Independent of r, K=768 Wb=128 has h-holes too: h=9 is 0/3 at both r=1 and r=2, and
h=16 inflated to 107 ms in one 12-sample cell. Consistent with §23 — the envelope is not a
single-axis H ceiling.
Also measured, and relevant to anyone sweeping this part: at Wb>=256 it refuses h<=12
outright (K=512 Wb=256 h=8/10 → 0/6, h=12 → 1/6; Wb=320 h=8/10 → 0/6, h=12 → 2/6, h=15
flaky). The search's 4-wide window never descends that far, which is a property of the window
worth keeping, not a coincidence to spend.
gam = 3*B/Pp is a RATIO of a host cache-scan term to a per-program term that is partly DRAM.
B should transfer (A76 at 2400 MHz on both boards); Pp should not (.236 is LPDDR4X at 2112,
.235 LPDDR5 at 2400), so .236's A should be LARGER and its gam SMALLER than measured here.
.236 was NOT measured for this entry. What its own reported behaviour constrains: at K=1024
M=128 Wb=128 tcrit=6 its model declines H=15 by a margin of 2 while H=15 is the better
geometry there, and cost(15) < cost(16) requires gam > 2.25 — which 2*Wmax = 4 satisfies and
the shipped 1.25*Wmax = 2 does not. So the two boards agree on the DIRECTION even though the
constant is per-part, and 2*Wmax (rather than 3*Wmax, which fits .235 equally well) is the
choice that leaves the most room for a slower-DRAM part.
If a cross-board conflict ever appears, do not re-tune the constant. Measure B and A once
per part at init — four cells is enough (two h at fixed g for B, two g at fixed r for
A), ~10 ms — and carry gam = 3*B*tcrit/A as a calibrated per-part term, the way the .orkpack
footer already carries the decode route.
- "at
Wb<=128 the tail scan is too narrow to pay (measured null)" — withdrawn. AtWb=128 the window optimum is worth −3.4 % to −13.7 % at K=512 and −1.7 % to −5.0 % at K=768. The null was read at M where the search does not move, i.e. it was an A/A against itself. - "at
Wb==64 it is a small loss" — withdrawn: K=512 M=128 measures h=32 at 1325.3 µs against h=31 at 1308.1 µs, a −1.3 % GAIN. The gate still excludesWb=64, because it is unmeasured on.236, not because it loses. - §24's "2.7 % at K=768, 3.3 % at K=512" understates the cell gain. With 72 samples K=768
M=128 is −3.4 %, and K=512 at
Wb=128 is −8.0 % at M=128 (−13.7 % at M=64). - Unrelated to int4, but it will cost the next agent board time:
test_speedfails the full suite on.235about three runs in four — 3621 / 3641 / 3658 µs against its 3600 µs limit — including on a PRISTINEaeb50f7control tree, while passing standalone (3371–3556 µs) and passing on the fourth suite run (3409 µs). It is an int8 multi-core latency guard and cannot be touched by any int4 change. NOT thermal: sampled 51.8–55.5 °C with the A76 never below 2400 MHz across a whole suite. It is the post-modelresidue jitter its own comment documents, against a threshold calibrated on.236that has only ~1–2 % of margin here.
§19.3's numbers were taken on d9c5d15. Three commits then landed on the same path before the pack
was committed — c2eb677 (tail-group balance heuristic), aeb50f7 (the completion gate: verify
fanned out over the pool, targeted dc civac in place of whole-slot kernel syncs, mode 6 fusing
verify and de-tile into one pass) and e6a4c99 (gam = 2*Wmax, gate from Wb >= 128, the r >= 2
guard) — so §19.3 could not be carried forward. Re-measured on 17dbb33 + the pack.
Board .236, 6.1.115-vendor-rk35xx-fencefix, DMC performance@2112 MHz and CPU performance
verified before and after. i4_e2_probe against an ABSOLUTE CPU int4×int4 reference, full-random
dm=1, ORK_I4_NOLADDER=1, 20 graded iterations per cell × four arm positions (ascending and
descending × two reps). ORK_I4_DENSE=0/1 on the same binary, so the arms differ by one
branch and one register value and nothing else.
| K | N | M | H | shipped µs | dense µs | ratio | §19.3 ratio |
|---|---|---|---|---|---|---|---|
| 512 | 2048 | 64 | 32 | 538.6 | 530.1 | 1.02× | 1.01× |
| 1024 | 2048 | 128 | 16 | 1180.5 | 1110.0 | 1.06× | 1.09× |
| 1024 | 2048 | 128 | 16 (Wb=64) |
1916.1 | 1693.0 | 1.13× | — |
| 2048 | 2048 | 128 | 8 | 2864.5 | 2475.1 | 1.16× | 1.14× |
| 2048 | 2048 | 100 | 8 | 2265.3 | 1964.0 | 1.15× | 1.14× (M=100) |
| 2048 | 512 | 256 | 8 | 1583.4 | 1377.2 | 1.15× | 1.18× |
| 2048 | 2048 | 128 | 16 (DBNK=2) |
2673.2 | 2211.8 | 1.21× | 1.17× |
| 4096 | 1024 | 64 | 4 | 1250.0 | 1225.3 | 1.02× | 1.01× |
Each figure is the mean of the four arm positions. 1920 graded iterations, every one bit-exact (96 cells × 20/20).
A/A control. Every arm position also ran a SECOND ORK_I4_DENSE=1 process at the same cell — a
cell where the geometry does not change at all. Worst A/A deviation 0.83 %, mean 0.30 %. So
K=512 (1.6 %) and K=4096 (2.0 %) are barely outside the noise band and should not be quoted as
wins; K ≥ 1024 is.
The pack and the completion-gate work do not cancel, and the win is not the one §19.3 measured.
The shipped arm itself got much faster in between (K=512 798 → 539, K=1024 1729 → 1181, K=2048
2983 → 2865), so the ratios moved in BOTH directions: K=1024 1.09 → 1.06, K=2048 1.14 → 1.16,
DBNK=2 1.17 → 1.21. Re-measuring was not a formality — carrying §19.3's K=1024 figure forward
would have overstated that cell by 3 points.
§19.3 deliberately left it on the stride-2 pack. It pays. test_moe_dispatch, PREFILL regime
K=2048 N=512 M=16 NE=32 (the qwen3.6 gate/up expert shape), 8 runs per arm in both orders
plus an A/A, grouped output graded against the absolute CPU reference:
| arm | shipped µs | dense µs | ratio |
|---|---|---|---|
ork_i4_mm_run_experts nc=3 |
3046.8 | 2689.3 | 1.13× (A/A 1.0008) |
ork_i4_mm_run_experts nc=1 |
5510.2 | 5484.4 | 1.005 (flat) |
ork_i4_mm_run ×32, separate (db path) |
5584.3 | 5212.0 | 1.07× |
CEILING, one M=512 run (db path) |
3084.5 | 2628.9 | 1.17× |
nc=1 is the degenerate single-chain arm that already loses to running the experts separately (0.95× there), not a production path; nc=3 is the one the MoE path takes.
That test's grouped check verified ONLY expert 0. It now verifies EVERY expert, because the
coalesce path stages each one at its own CUMULATIVE activation offset — precisely the arithmetic
the dense pack changes, and precisely what a single-expert check cannot see. All 32 experts were
bit-exact in all 24 runs. The mw path is additionally covered inside make test: the grouped int4
path (ork_i4_mm_run_grouped, examples/i4.c G=64/128 and test_slice_rescue's i4g-refuse)
routes through orki_i4_run_experts_bchain_db → bch_mw_worker, and reports dequant maxerr=0.
It is the single negative across both boards and it reproduces in both arm orders. On .236 that
same geometry is the 1.13× row above. It is not gated off, because:
- a per-geometry opt-out fitted from one die's measurement is exactly the modelling error §17.5 and
the
orki_i4_hcaphistory already document ("do not fit a closed form to one board's data"); - gating would fork the two boards' behaviour, making their numbers even less comparable than the DDR type already does;
-
ORK_I4_DENSE=0is already the per-process escape hatch, and it restores the shipped pack exactly (the A/A and the DENSE=0 cells above reproduce the shipped numbers in every arm position).
Every number here is at the BCHAIN matmul level. No end-to-end model benchmark was run for this
change, so there is no prefill/decode token-rate claim attached to it. make test ALL TESTS
PASSED on .236 with the pack as the default, tests/sbc_attest.txt CORE_SHA =
ffaaaa000a0a060ce14ad03312d2f382446c4fe032259871b4aa0a7db2fa6ad6.
The first sweep looped forever on one cell: its descending pass used for ((i=...; i>=0; i--))
while its wait_guard helper used for i in $(seq 1 20) without declaring i local, so the
guard reset the loop index on every call. Shell helpers in a measurement driver must declare their
loop variables local — an index clobbered by a helper looks exactly like a cell being re-run.