0098 the normal quantile is the third member decision 0095s rule publishes - CyrilB1531/lodestar GitHub Wiki
0098 โ The normal quantile is the third member decision 0095's rule publishes
Status: accepted ยท Date: 2026-09-10
Context
Decision 0095 published four numerical members and wrote the rule they were published under: the layer publishes what one caller needs, and publishing later is always available. Decision 0097 applied it a second time, for the chi-squared tail a log-rank test reports.
#569's Kaplan-Meier curve asks a third time.
Its confidence bounds are the ones lifelines reports, built on the log-log transform of the
estimate, and their multiplier is scipy.stats.norm.ppf โ a standard normal quantile.
Normal.Quantile exists in Lodestar.Stats.Internal, and 0095 listed it by name among the members
that stay internal.
The substitute was tried first, and measured
Student converges on the normal as its degrees of freedom grow, and
Distributions.StudentQuantile is
already public. StudentQuantile(0.975, df) at a very large df is therefore the obvious way to
avoid publishing anything. It does not work, and the reason is not the one expected:
df |
StudentQuantile(0.975, df) |
relative error against norm.ppf(0.975) |
|---|---|---|
| 1e6 | 1.9599663569861794 | 1.2e-6 |
| 1e7 | 1.959964222613035 | 1.2e-7 |
| 1e8 | 1.9599639666317055 | 9.1e-9 |
| 1e9 | 1.9599636901568092 | 1.5e-7 |
| 1e12 | 1.9598992657185477 | 3.3e-5 |
Accuracy has an optimum rather than a limit. Below 1e8 the convergence to the normal is still
incomplete; above it the bisection inside the Student quantile loses more than the convergence
gains. The floor is about 9e-9, and the log-log transform amplifies that into the seventh digit
of a survival bound โ past the 1e-9 the corpora are compared at. The failure was found by the
frozen corpus, not predicted.
Decision
Distributions.NormalQuantile becomes
public. Lodestar.Stats goes 0.3.0 โ 0.4.0. It answers to about 1e-15 across the range, seven
orders of magnitude better than the substitute.
Two details carry over from 0095's experience of publishing StudentQuantile:
- It is the quantile, not the inverse survival function. The internal helper solves
P(Z > z) = pand carries the opposite sign. The published member negates โ the distribution's symmetry, not a correction โ exactly as 0095 found and fixed for Student. - The median returns positive zero. Negating the helper's exact zero would publish
-0.
Everything else in the layer stays internal, unchanged: RegularizedIncomplete, LogGamma,
RegularizedP, RegularizedQ, Erfc, Normal.Sf, the finite-sample Kolmogorov distribution,
PartialPivotLu, JacobiSvd and the DenseBlock helpers.
Options refused
Approximate it with a large-df Student quantile. Refused on the measurement above: it cannot
reach the tolerance, and burying a 9e-9 error inside a confidence bound is the kind of avoidable
numerical divergence decision 0081 exists to
prevent.
Re-derive a normal quantile inside Lodestar.Survival. Refused for the reason 0097 refused the
same move for the chi-squared tail: a second implementation of a function this repository already
has, with nothing making the two agree.
Consequences
- One member, one reference page, one row in the tails index, one use in the packaging sample, and
six cases added to
tests/oracles/stats_distributions.json, compared relatively. - A test pins the substitute's failure rather than only the member's success: the Student quantile
at 1e8 degrees of freedom is asserted to differ from this one at
1e-12and agree at1e-7, so the measurement above cannot rot silently. Lodestar.Survivalwaits on 0.4.0 reaching the feed, as it waited on 0.3.0. Same reason, same shape:src/floors on published versions (decision 0012), and CONTRIBUTING's Working across two packages refuses to merge two independently released packages as one.