0054 the payload buffer is pooled after all because the collection is the cost - CyrilB1531/lodestar GitHub Wiki
Status: accepted ยท Date: 2026-08-29 ยท Amends: 0053
0053 refused renting the payload buffer. It measured two columns โ 37 069 648 โ 16 480 488 bytes allocated per load, and 33 554 432 bytes held by the pool for the life of the process โ and decided on them.
It never measured the third. No timing of the pooled path exists anywhere in that lot. The 8.1% it quotes is #433's warm-heap figure, which is the value of pages already committed between a cold process and a warmed one. Pooling does something else: it removes the allocation, and with it the collection the allocation provokes. 8.1% was never its ceiling.
bench/Lodestar.Text.Benchmarks -- pool-cost, on a hosted runner, interleaved in one process,
both states touching every page. Against an uninitialized allocation, not a zeroed one: the
read path uses Buffers.AllocateUninitialized, so charging the pool's rival for a memset the
code does not perform would have inflated the saving by the whole zeroing.
| median | min | max | |
|---|---|---|---|
| allocate 20 589 007 bytes | 1.783 ms | 0.071 | 2.756 |
| rent and return | 0.042 ms | 0.038 | 0.068 |
42ร, and 1.74 ms per load. Against heap-warmth cold's 18.101 ms on the same class of
machine, that is about a tenth of a load.
The allocation is not what costs. Its minimum is 0.071 ms โ as cheap as the rent. The median is twenty-five times that because a large-object collection fires on most iterations. On the shared container the same rows read 0.14 ms at the minimum and 8โ135 ms at the median, which is the same finding with the noise turned up. This is exactly the mechanism 0053 printed and did not price: collections 4/4/4 against 1/1/1.
The payload is pooled. EmbeddingIndex.Load(Stream) rents from ArrayPool<byte>.Shared and
returns the buffer once FromPayload has parsed. The code is #435's
unchanged, restored from a644bc0.
The residency is the price, not the refusal. The pool still rounds a 20 589 008 rent to
33 554 432 and holds it for the process. 0053 read that as decisive because it weighed it against
nothing; weighed against 1.74 ms and three collections per load it is a trade this library takes.
Throughput is what it publishes โ embedding_index_load sits about 4ร behind numpy.load โ and
resident memory is not.
What 0053 got right and this keeps. The rent lives at the Load(Stream) call site, above
FromPayload, because the public Load(ReadOnlyMemory<byte>) overload reaches FromPayload too
and parses the caller's memory: pooling one line lower would return a caller's buffer to the
shared pool. And the slice is taken from the byte count read, never from the buffer's length,
which PooledPayloadTests pins โ those tests were kept by 0053 for this attempt and now guard a
live invariant rather than a dormant one.
What would reopen this. A caller that loads one index in a short-lived process and is measured on peak resident memory rather than on load time. Nothing published here is.
-
TryReadDeclaredLengthbounds its read loop by the declared length, notbuffer.Length. Those are the same number only while the array is allocated exactly; a rented array is at least as long as asked, and the old bound would have read past the artifact. -
A stream with no declared length is not pooled. The growable path sizes itself as it reads
and owns the buffer it grows, so
RentedPayloadcarries it owning nothing andDisposeis a no-op. One shape at the call site rather than a branch to get wrong. -
The buffer is not cleared on return. Zeroing 20 MB costs the memset this decision exists to
avoid, and
ArrayPool<byte>.Sharedis process-local, so the bytes were the caller's own artifact already. -
pool-costis committed, so the number this turns on is re-runnable rather than remembered. It is a diagnostic, not a nightly row:bench-map.jsonnames it indiagnostics, which is where a subcommand the nightly deliberately never runs is declared. -
An unknown subcommand now exits 2. It used to fall through to
BenchmarkSwitcher, which prints its menu and exits 0 โ so a typo in a dispatch produced nine rounds of nothing and a green run. That happened while taking this measurement.