fp16 Fused Gate+SiLU — ✅ CALIBRATED (mean|err| ~0.08 / ~1%), gated OFF (speed)

Status: ✅ CRACKED — the fp16 matmul + fused SiLU produces correct silu on-NPU (mean|err| ≈ 0.08, max 0.75 over silu[−8,8], ~1%, on par with the int8 fused SiLU), no wedge. Kept gated OFF: the fp16 matmul is ~3.3× slower than int8, so it's a measured net-loss today (tools/f16_gate_bench). Built out as the "end-goal" higher-precision on-NPU gate for a future pipeline (batched-verify / int8-win regimes) where precise activation with no int8 quantization pays off. Commits: ork-driver 27e7cf1 (primitive), 8cbceab (calibration crack). Probe: tools/silu_f16_calib.c. Primitive: ork_mm_run_f16_silu / set_f16_silu.

Why this matters

The int8 fused SiLU quantizes the silu output to int8 — the ablation (ORK_GATE_ABLATE) showed that is the entire PPL gap of the on-NPU FFN chain (fp32 silu = baseline PPL; int8 silu = the ~16-vs-9 loss). A fused gate that emits silu at fp16 precision removes that quantization while keeping silu free-on-NPU. The open question was whether the flying-mode matmul SDP can drive the LUT with a non-int8 output at all.

Hypothesis (supported)

That an all-fp16 fused op (fp16 matmul + SiLU LUT, keeping REGCMD's native fp16→fp32 output CVT) would run on-NPU where int8-in→fp16-out wedges. Earlier attempts to force non-int8 output on the int8 matmul program (0x4010 PREC=1/2, int32 bit-31) all soft-reset — a proc-precision mismatch (int8 processing vs a float output CVT). The all-fp16 graft has no mismatch. This held: it runs, no wedge.

The two-part structure (answers "model or board?")

1. idx(acc) → LUT-index mapping = BOARD (the SDP's fixed index hardware; measure once per config).
2. LUT curve values: int8 = per-layer (each layer's quant scale changes the acc→gate mapping); fp16 = UNIVERSAL — the accumulator is the real gate value, so silu(gate) is one curve for every layer and model. Calibrate once, never per-layer. (A genuine fp16 advantage.)

The blocker and the crack

Ramp-measuring idx(acc) (tools/silu_f16_calib) exposed a hardware asymmetry:

acc	idx behavior
acc = 0	idx = 514 (= C0 from idx_off=0xffffc000)
acc < 0	idx decreases toward 1 — spreads across the LUT
acc > 0	clamps at 514 — one index, curve unrepresentable

So positive gates (silu's important half, silu≈gate) all mapped to one LUT entry. Register probing:

register	role (fp16 program)	effect on index
0x4084/0x4088	output gain R (out = R·LUT[idx])	none (output-only)
0x4044 (za, BS_ALU_OPERAND)	pre-LUT bias in int8	none here
0x4064	fp16 index-scale param	default = usable spread; 0xffff7dc8 (standalone silu's value) collapses it
idx_off 0x4110	index center C0	shifts center, doesn't fix the positive clamp

The crack: since only negative acc spreads, negate the gate — pack the gate weight as −S·W so a positive real gate becomes a negative accumulator (spreads), and negative real gates become positive acc → clamp to ~0, which is exactly silu(neg)≈0. Scale S≈24 maps [−8,8] onto the ~513-entry negative half. Then the standard 2-pass build: measure idx(acc) with a ramp LUT, fill LUT[idx] = silu(−acc/S)/(R·out_scale).

Result (tools/silu_f16_calib, RK3588, K=512 N=64)

S sweep (negated gate, idx_off=0xffffc000, cfg4068=0x56391100, default 0x4064):
  S=24 : mean|err|=0.0775  max=0.7530
  S=30 : mean|err|=0.2298  max=2.2690
  S=38 : mean|err|=0.4109  max=3.2961     (larger S over-spreads -> clamps)
per-point (positive gates correct, negative lobe clamps to ~0):
  true=+2.0  npu=1.757  ref=1.762
  true=+4.0  npu=3.920  ref=3.928
  true=-2.0  npu=0.000  ref=-0.238   (silu neg lobe, clamped ~0)

S≈24 → mean|err| 0.078, max 0.75 over silu[−8,8] (~1%). silu's shallow negative dip (min −0.28) is the only real loss (clamped to 0); the dominant positive lobe is accurate.

Speed (why it's gated OFF) — tools/f16_gate_bench, gate M=128 K=2048 N=6144

int8 matmul                          :  4640 us
fp16 matmul                          : 15477 us  (3.34x int8)
CPU silu (M*N)                       :  6158 us
BASELINE gate (int8 mm + CPU silu)   : 10798 us
FUSED fp16 gate (~fp16 mm)           : 15477 us  -> 1.43x SLOWER

Correct, but a net-loss today: the fp16 matmul penalty (+10.8ms) exceeds the CPU-silu it removes (6.2ms).

Register roles (fp16 fused SiLU, RK3588) — reference

• 0x4010 = 0xa8000002 — fp16 output CVT (kept from REGCMD; do not override to int8 0x44e0).
• 0x40c0 = 0x40 (2-byte elem), 0x4050 = 0x36e (fp16 row stride) — from REGCMD.
• 0x4084 = output gain R; 0x4088 = shift. Index-independent.
• 0x4064 = index scale — keep REGCMD default (0xffff7dc8 collapses index).
• 0x4110 = idx_off = C0 center; 0x4068 = cfg4068 (0x56391100). 0x4044 (za) inert.
• Submit: LUT-load prologue enable=0x18 regcfg=1097, then matmul+silu enable=0x1d regcfg=REGCMD_N.
• fp16 weight lives in w->Bb (single tile: Sk==1 K≤2048, Sn==1 N≤nmax); Bf is int8-only.
• Env knobs on set_f16_silu: ORK_F16_C4064 / ORK_F16_ZA / ORK_F16_R84 / ORK_F16_R88.

Upper LUT bank — CONFIRMED unreachable for fp16 (2026-07-05, tools/f16_bank_probe)

The RK LUT is 2 banks (neg 0..511, pos 512..1023); the int8 fused path uses BOTH (idx=in+512). The fp16 path only ever reaches the lower bank — the negate hack exists because positive acc hard-clamps at idx ≈ 514. Question: is the upper bank reachable under some never-checked register config? Answer: no. Measured index vs acc (default): idx = 512 + 3.07·min(acc,0) — negative acc spreads 1..512, positive ReLU-clamps at 514. Swept every accessible register — none lift the clamp:

register(s)	result
0x4044 za / 0x4064 idxscale / 0x4068 cfg / 0x4108/0x410c LUTcfg	no effect on the +clamp (some collapse the spread entirely)
0x4110 idx_off	controls the negative-spread slope, not a center offset; +clamp stays at 514
0x4070/0x4004/0x411c/0x4128/0x412c	no unlock (0x4004 bit0 wedges; kernel soft-resets & recovers)
0x4010 output CVT (WEDGE-PRONE)	either wedges, breaks output (R~0), or no index effect — as expected (it's post-LUT)

Conclusion: the one-sidedness is intrinsic to the fp16 flying-mode accumulator→index datapath, not register-configurable. fp16 fused SiLU gets ~512 entries (lower bank), which is already 2× int8's 256 — so the fp16 error (~0.055) is not resolution-limited (it's fp16 weight/acc rounding + the single-scale compromise). The upper bank's only real gain would be silu's negative lobe (currently clamped to 0, max ~0.28 in a low-weight region). Not worth the CVT wedge risk. The negate hack is the correct, complete approach. RE knobs added for the record: ORK_F16_C4004/C4010/C4060/C4064/C4070/C4108/C410C/C411C/C4128/C412C + ZA.

In-model integration + bounded-Gmax fix (2026-07-05) — coherent, still gated OFF

Built ork_mm_build_f16_silu_lut() (bakes the negate/scale, returns the measured-idx LUT + S + out_scale) and wired ORK_FFN_GATE_F16 into the ggml-ork hybrid. First in-model run = PPL 25k (garbage). Root cause was NOT a wiring bug: S = 296/Gmax, so wide-range layers (measured gmax up to 130 on Qwen3-1.7B) drove S tiny (~2.3), crushing the bulk of the gate distribution into a few LUT-index entries near the center → those layers emit near-garbage silu, and one bad FFN layer nukes PPL.

Fix (commit 0de5e28): bound the effective Gmax + tune the acc target. ORK_F16_GCAP (default 40) caps Gmax so S stays in the validated spread band (gates above the cap clamp to silu(cap), ~linear so cheap). ORK_F16_ATGT (default 150, replaces the hardcoded 296) sets S = ATGT/Gmax = the acc range; 296 was over-spreading small-Gmax layers (gates near Gmax hit the +clamp edge). tools/f16_gmax_sweep result:

ATGT	bulk|err| @Gmax8	@Gmax20	@Gmax40	@Gmax130
296	0.41 (over-spread)	0.057	0.055	0.036
150	0.055	0.061	0.065	0.034

ATGT=150 gives uniform bulk|err|~0.055 across the whole gmax range (8→130) — the primitive is now COHERENT in-model. Still gated OFF: fp16 matmul 3.3× int8 (speed-dominated by the CPU-fp32 silu sprinkle, which is exact AND ~2.3× not 3.3×). This makes the fp16 gate a ready primitive for a future fully-on-NPU pipeline (batched verify), not an enabled path.

TODO to enable (future)

build_f16_silu_lut() exists (see above). Remaining: static-const the universal LUT (fixed S, RK3588-specific, like REGCMD_SILU_LUT) and enable only where the fp16-gate cost is hidden (batched verify) or the precise gate unlocks a downstream win. The upper-bank crack (above) is the way to make it accurate (not just coherent) — but it's walled off, so single-region + bounded-Gmax is the ceiling.