JWT cache cost preview

Before adding a tiny hint model, we measured the cost of repeating the same work. This separate Go execution found that safely reusing signature verification and decoded claims made fixed-order search faster. Model calls, new training, GPU calls and paid calls were zero.

What we observed

We paired uncached and cached paths for the original 160 observations. Every pair matched in acceptance, error classes and claim fingerprints, and matched the previous original results. Four controls then ran over the same four requests for three rounds; all 48 searches completed.

Control	Four requests × three rounds, total
Fixed source order, uncached	5.141583 ms
Fixed source order, cached	2.711623 ms
Predeclared candidate first, uncached	2.428458 ms
Predeclared candidate first, cached	2.593877 ms

Caching made the fixed-order aggregate about 1.90× faster. When the first candidate matched, caching was about 6.8% slower. First candidates [4, 5, 2, 0] were declared from request contracts before observing outcomes; they are not model predictions.

What the cache does

A five-entry array is cleared for every request, control and round. Authentication and decoded claims are reused only for the same entire token, public key, method, decoder and source. Audience slices and date objects are deep-copied, and every candidate gets a fresh policy check. Final Token.Valid and policy results are not cached. Unexpected errors, mutation and deadlines remain unknown and stop search.

Across four requests, fixed order changes 43 parses into 20 authentication parses plus 43 policy validations. Predeclared-first still needs 20 parses with zero cache hits, so caching adds overhead.

What this does and does not establish

All three rounds followed the 160-pair check in the same warmed process. They are not 48 independent new tasks. These four requests are exposed development_validation for the whole JWT lineage, not a 2,400-task evaluation or newly admitted training data.

Whole-process peak RSS was 14,532,608 B, about 13.86 MiB; memory was not attributed to modes or models. Whole-child CPU was user 29.161 ms + system 8.955 ms, and main took 23.256416 ms. Start through pipe/wait cleanup took 608.614250 ms: this does not isolate startup or define a portable constant. Main excludes import initialization and output marshaling/writing.

Ordering benefits after caching varied by round and were negative once. No reliable feature-extraction/inference budget or Codex saving is established. Next, compare inexpensive nonlearned BM25/Jaccard retrieval under the same checking conditions.

Inspect the evidence

With Go available, run from the repository root. This reads saved evidence without executing the original JWT API, a model or training.

go run ./experiments/short-claim/next-cohort-jwt-audit/cost-replay

Pinned source and detailed usage also explain optional new native execution, which is separate from replay. Owned code and fictional inputs are Apache-2.0; copied JWT code is MIT with its original notice preserved. See SDK notices.

Publication receipts are linked after verification: PR · CI · Hugging Face.