Stored-truth nonlearned utility58 plan

한국어 · Prior group audit · Public behavior57

This is a pre-observation development diagnostic. New truth, independent requests, roles, fits, models, weights, paid calls and protected-final reads are0. Reuse stored truth for72 parents to assess ordering headroom. Do not execute candidate sources again or merge57's62 operations into training labels. Proceed within the user's existing authorization without adding human approval.

Question

Before training a small hint model, determine whether a better order could reduce independent checks. Compare four fixed controls using only request/candidate prose and calculate an upper bound assuming known answers. This bound is not achieved model utility, actual candidate execution, CPU/RSS/time or LLM/token/cost savings.

Frozen input	SHA256
Existing48 parents	0fe97dd65606f4239ef9187f1fc4d9c8dc2fcaebf342612b2071220896b92df0
Additional24 parents	0d2bf6980353cefc4327af165c0f60a6455c1feea920f354dc621019ce3ed21e
Stored truth/groups	4128400c792d2151955a1d98f7d35aca6112d097a3ac20aa280a7357996636c4

The72 parents/216 captions comprise34 answerable,17 no-answer and21 unknown. Preserve12 requests with2 candidates,48 with3 and12 with4. Existing17 total/16 labeled connected groups already pass the15-group operating floor; do not add a15-repository requirement. Preserve the single synthetic English workflow limitation and prior unknown/pending evidence. Previously seen56a rankings for48 parents remain separate from this new72-parent diagnostic.

Binding and feature boundary

Go storedaudit.Bind first checks three exact raw SHA/byte pins, then binds ordered parent/candidate identities and counts, acceptable indices, states, vector records and group membership. Raw input SHA differs from canonical typed-dataset SHA. Reject duplicates, bounds violations, contradictions and incomplete inputs with fixed diagnostics; never relabel unknown as false/no-answer.

Text contains only raw request and ordered candidate prose. Truth, source/prototype/contract/group/role and document filenames are separate metadata. Scorers receive predictable local candidate IDs and fixed provenance, neither used as features. Preserve512-byte valid UTF-8 and the existing32-normalized-word contract without truncation/rewriting. Preparation binding tests and literal permutation controls are neither new source truth nor actual72-parent ranking observations.

Costs and controls

The fixed order is fixed_order, bm25, lexical_ordered, narrow_rule. Unsupported prose uses the existing BM25 fallback; report fallback out of all72 including unknown. Never rewrite requests into behavior-v1: after observations. Retain every candidate and use original order for ties.

• Answerable: checks through the first acceptable candidate; multiple acceptable answers are allowed. Oracle still costs1.
• No-answer: every candidate is checked by both controls and oracle.
• Unknown21: retain the full72 denominator, relations and reasons, excluding these rows from cost/Top1/Top3. Excluded zero is not evidence of zero actual checks.
• Report known51 and answerable34 costs separately; Top3 easily saturates with2–4 candidates.
• Publish per-parent/group costs and a macro over the16 groups with known requests as additional diagnostics, not replacements for the global5% condition.

The best control is one fixed baseline with the lowest known51 total, never a per-parent mixture of favorable controls. Ties use the array order above. Necessary headroom is (best_known_checks-oracle_known_checks)/best_known_checks, with an exact integer test for at least5%. Below5%/already optimal stops training on these data. Passing cannot certify learnability or training eligibility.

Freeze and limits

Pin9 non-test Go sources,13 support files, three inputs/execution plan and actual binary SHA/bytes, Go1.27.1/CGO0/platform. Source and input freezes are separate40-hex Git commits, with every actual blob compared before ranking. Reject extra Go sources, unknown/duplicate/extra-array/noncanonical plans and changed binaries.

One official diagnostic, zero retries. Reserve a new directory and exclusive result file before execution. This is not a global one-shot guarantee; keep an external official ledger. Distinguish dispatch, validation, attempted Baselines calls, completed control rankings and completed rows. Preserve partial/error records and report writes failing after calls. Each Baselines invocation returns four controls: the success target is72 calls/288 rankings/72 rows, not additional independent samples.

Each input/source/result is bounded to1MiB and binary to64MiB. Git blob reads have per-file5s deadlines and bounded stdout. A10s cooperative evaluation deadline checks between bounded rows and at completion, without detached workers. GOMAXPROCS1 and256MiB Go heap soft limit are settings, not measured RSS/CPU/GPU/latency. New resource claims, cache, SIMD and lock optimizations are outside this diagnostic.

Freeze the CI replay comparison before official observation. Orders, integer check counts, states, reasons, denominators and the remaining evaluation record must match exactly. Only unverified floating scores allow Linux/macOS rounding differences: both values must be finite and satisfy abs(actual-saved) <= 1e-12*(1+abs(saved)). Compare scores first, remove only those fields, then compare everything else exactly. Pin the original result-file SHA separately; score tolerance never permits changing the report or accepting different orders. CI replays do not add official attempts or independent requests.

Next decision

Preserve no_roles_plan and synthetic_single_pipeline. Beyond necessary headroom, examine prose fidelity, behavior/language coverage, authoring diversity and public-source transfer relations. Freeze whole-component train/validation/calibration memberships and a separate training execution plan; do not pick favorable validation groups after seeing utility.58 has roles0/fits0/training_ready=false and cannot activate production.

The2400 distinct protected-final requests/domain goal is for later final utility/generalization claims, not a universal minimum before development fitting. Variants, reversals, translations, literals, repeats, seeds/models do not fill2400. Existing protected final/CoSQA reserve remains unread/unreassigned. FP32 primary, INT8/PTQ children and separately trained ternary STE siblings remain future qualified comparisons.

Keep model binaries outside Git. Reuse existing original public Apache-2.0 data and pin LICENSE; preserve56a/b/c/d/e/57 records. Freeze Go evaluator/plan first, add results and CI regression guards separately, and link Wiki/issues only after actual CI success and automatic merge.