Limite RL health and lessons

Evidence snapshot: 2026-10-07. This page summarizes recorded experiments; it does not report current node occupancy, queues, or completion estimates. The detailed operational chronology is retained in the private archive.

Capability outcomes

Read the matched SFT comparison for lineage, geometry, parameter differences, SFT data issues and the full evaluation contract.

Continuation	AIME26 before	AIME26 after	Interpretation
Normal SFT10B → RL400	15.83%	12.50%	No observed gain
Simplicial SFT10B → RL400	20.83%	20.83%	No observed gain
Released Violetto → RL142	92.50%	83.33%	Observed regression

Each endpoint uses 30 problems × 4 seeded responses. The paired problem-bootstrap 95% interval for Violetto's change is [-18.33, -1.67] percentage points; for simplicial minus normal at RL400 it is [0, 18.33]. This uncertainty covers resampled evaluation problems, not variation across training seeds.

Finite gradients and valid checkpoints establish execution health, not improved capability. In the last 100 steps of the historical paired RL runs, 84–86% of response groups had no reward contrast.

Distinguish three causes of poor completion

• Historical RL imposed a 16,384-token completion budget and a soft length penalty above 12,288 tokens. Of 111 correct released-Violetto benchmark responses, 69 exceeded 16K, 31 exceeded 32K and nine exceeded 64K.
• The full-context benchmark uses 131,072 minus actual prompt length. An audit of 959 responses found no hidden 16K/32K cutoff or budget/EOS inconsistency.
• Historical SFT dropped most document endings. High SFT native-context exhaustion frequently accompanies repetition; a longer limit alone does not repair that learned behavior.

The native-context Violetto successor removes extra length/unfinished penalties and records each response budget. This is a distinct protocol phase and has no claimed benchmark improvement in this snapshot. The original direct-base pilot was different again: 8K response cap, only one applied update in 15 steps, and 2/60 groups with reward contrast.

Correctness and performance qualifications

Change	Evidence and limit
Native grouped-query cached decoding	Whole-model likelihood/replay checks; graph cache reuse and finished-response compaction
Replay padding trim and flat-work skipping	Preserve loss normalization; zero-active distributed ranks remain synchronized
Actor/learner coordination	Drain retained actor work and use CPU rendezvous before learner GPU collectives; maximum policy lag one
Checkpoint staging	Preserve model, optimizer, scheduler and all-rank RNG; verify published payload hashes
Native full-context replay	Container cuDNN reduction with original full Q/K/V geometry, decoder checkpointing and chunked output head
131,072-token admission	31.05 s replay/update probe, 123.44 GiB peak allocated on B300 with resident actor/cache; not end-to-end RL step time
Distributed resume	Eight-rank fixture from step142; exact state reload and next-update equality; synthetic fixture never used as a production checkpoint

FA4 and tiled local replay candidates did not meet the declared full-model continuation tolerance despite close individual-layer agreement. They remain experimental; tolerance was not relaxed to admit them. The final native path's 2053-token probe had maximum selected-log-probability error 3.8e-6 and gradient relative L2 error 0.00561.

Next experiments

1. Qualify a matched SFT repair retaining document endings; report overlap and short-document inclusion explicitly.
2. Measure training-only reward contrast and adjust curriculum before scaling rollout volume. Do not use benchmark answers to select training examples.
3. Batch distinct prompt groups and reduce whole-prefix repetition rescanning, preserving response RNG, policy lag, EOS, budgets and checkpoint semantics.
4. Validate benchmark rendering against the publisher's evaluation harness. Its current Base profile is few-shot; historical zero-shot Base results are not a reproduction of that protocol.

The publisher now provides an evaluation harness. Its critic notes describe SFT initialization, offline value fitting, online actor–critic training, and informative mixed-outcome groups. They do not disclose a complete actor recipe or establish equivalence to our GRPO/DAPO experiments.

Raw run evidence remains in team storage. Public aggregates are in recorded results.

Matched restart contract, 2026-10-09

The new matched protocol restarts normal and simplicial from their respective matched 10B SFT weights, with fresh full-weight RL optimizers and identical data, seed, curriculum, sampling and loss settings. Each variant uses eight B300 GPUs. The response budget is the native 131,072-token context minus the actual prompt; an exhausted budget is recorded explicitly and never relabeled as a natural EOS.

The root verl submodule pins XiaomiMiMo's code. The native Limite executor loads its repetition detector directly and records the component hash. This is reuse of upstream components; the complete verl Ray/FSDP execution stack is not qualified for these custom adapters.

Admission separates exact graph/cache migration checks and independent FP32 attention oracles from BF16 differences between decoding and uncached replay. The denominator is the actual sampled actor probability. Distribution-wide importance-ratio clipping mass, finite gradients, fresh-Adam updates, full-context resident memory, and distributed checkpoint/resume behavior are checked separately. Full-context resource probes passed for both variants with about 184 GiB peak reserved memory per GPU; these disposable probes do not establish production learning health or benchmark improvement.

For complete sampled trajectories, health uses clipping frequency, effective sample size and mean importance weight, following the pinned upstream rollout correction diagnostics. The recipe requires at most 5% clipped tokens, at least 95% effective sample fraction and mean weight between 0.5 and 2. Individual ratio extrema remain reported: a single low-weight tail token is not an importance-sampling correctness failure. Short-canary precision targets are kept separate from these full-trajectory health criteria.

The conditional distribution check averages exact full-vocabulary clipping mass over active token rows within each canary. This estimates its expected clipped-token fraction and follows the summed-token objective; it is not the worst individual position. Both actor and native mean mass must remain below 5%, every conditional effective sample fraction must exceed 95%, and normalization and exact cache/math checks must pass. Worst-position mass and the earlier failed worst-position verdict remain recorded. These diagnostics do not bound gradient bias for arbitrary advantages or establish capability.

Historical qualified launch, 2026-10-09

Both previous continuation queues were cancelled. Final qualification passed for both variants, including the complete 131,072-token resident backward and update, exact graph/cache controls, gradient/update numerical checks, and save/reload/resume with optimizer and RNG state on all eight ranks. Fresh RL controllers launched at 04:36 UTC, with eight B300 GPUs per variant, from their matched 10B SFT checkpoints. This entry records the qualified launch while startup was in progress; the first real production update had not yet been confirmed.

Actual training-data canaries used four complete responses per variant. Every response ended naturally with EOS; the native context budget remained enabled. The behavior/replay checks measured:

Variant	Completed response tokens	Tokens outside PPO ratio [0.8, 1.2]	Effective sample fraction	Mean importance weight
Normal	85,163	0.135%	99.913%	0.999947
2-simplicial	130,435	0.108%	99.914%	0.999865

These are execution and importance-sampling diagnostics from training data. They establish neither a benchmark score nor an architectural advantage. Long responses remain possible, and live learning health requires monitoring actual production updates.

The qualified public implementation and matched recipe were published through PR #4. Public squash commit aa2add2f11af3508559109c2ab45964bd3cfe22f has exactly the same Git tree (45e651e749c1ba47fb326a57f8867533dbff574b) as immutable run revision 5539f5f05927147b0fef3ff0879c2c4e868c2775. All 862 tracked snapshot blobs and the pinned upstream submodule were additionally verified. Private receipts preserve that mapping and runtime evidence without publishing staging files or transient local history.

After-update rollout scheduling

The v5 matched recipe explicitly sets rollout_prefetch_schedule: after_update with actor/learner overlap disabled. The previous no-overlap schedule generated and drained the next batch before applying the current update. Its first normal production update took 11,751.5 seconds, with 2,436.5 seconds in local next-batch drain and 4,307.0 seconds in the subsequent host rendezvous on the reporting rank. These durations include rank imbalance and are not an architecture comparison or a forecast of steady-state speedup.

The revised schedule consumes the current batch and completes the host rendezvous before replay; the next generation starts on demand after the optimizer publishes its policy snapshot. A retained legacy prefetched batch is consumed once with its original probabilities and RNG provenance. Future batches use the newly published policy. The 64-response budget, objective, native context and checkpoint state format remain unchanged. Recipes without the new key retain the historical schedule.

This revision removes unnecessary work before an update and avoids checkpoint drains of a future generation. It does not reduce the number of responses per update or the cost of long responses; throughput gains require a measured post-relaunch comparison. Production qualification and migration receipts are kept separately from this source-level description.

Scheduler qualification and publication, 2026-10-09

Both variants passed fresh eight-rank training and resume controls, including flat-batch/evaluation transitions, exact optimizer and per-rank RNG restore, checkpoint reload, and next-update equality. Migration from completed legacy prefetched batches also passed for both variants: the retained batch was consumed exactly once with its original probabilities at policy lag one, and new after-update checkpoints had no future actor batch pending. Unchanged attention, decoding, objective, optimizer, checkpoint, and native-context resource controls were inherited through recorded component and source checks. These are execution qualifications, not a benchmark improvement or a measured steady-state throughput gain. The prior production runs were gracefully saved and paused before the successor admission. Live successor status stays in private receipts.

The scheduler revision was published through PR #5 as public main commit bd694fb. Its Git tree 62c128b925ab7a7b79d410e983d5d1faf7246ca0 exactly matches frozen qualified revision b44c34cd72625f8e777f3319a8cb5e4d3fe88d29.