0003 the package layout tiers boundaries and edges - CyrilB1531/lodestar GitHub Wiki
Status: accepted ยท Date: 2026-09-24
Seventeen records decided, one lot at a time, where a capability lives and what it may carry:
0016 put the metrics outside the text package, 0071 moved CsrMatrix out from under it, 0076
stated the tier rule, 0081 kept a numerical layer internal, 0089 opened the interop tier, 0095
published four of that layer's members with 0097 and 0098 publishing a fifth and a sixth under
the same rule, 0096 gave ordinary least squares a package while 0111, 0119 and 0124 refused
one to the generalized linear model, the explained variance and the Cox model, 0100 and 0101
(amended by 0103) added the last two satellites, and 0138 and 0139 took the two most recent
edges. Read singly they answer one lot each; read together they are one axis, and this record is
that axis.
This is the record's second text, written in numbering epoch 3. The first, dated 2026-09-20,
drew Lodestar.Abstractions' line by exchange โ it held "the types packages exchange" โ so a
public enum or options record no second package happened to name stayed where it was declared, and
an audit on 2026-09-24 over the net10.0 assemblies of 9f9406c5 found 94 such types across thirteen
packages. That line is replaced by (e) below. The same pass drops the table of edges the first text
carried: it stated sixteen while the graph held seventeen, and an immutable record cannot follow a
graph that grows. #1103 carries the rewrite;
the first text reads at 9f9406c5.
tools/check_nuspec_dependencies.py's EXPECTED is the authority on the shipped graph, because
dotnet pack derives a package's dependencies from what restore resolved rather than from anything
a human wrote. Each edge is asserted there per target framework and per version range, and
CLAUDE.md's count follows it under tools/check_claude_md_packages.py.
(a) A capability earns its own package when its audience is distinct, and otherwise it is a namespace
A split is licensed by a distinct dependency profile, a distinct audience or a distinct release cadence, never by tidiness, and the audience test is the one that decides most lots: a package is earned when a caller who wants the new capability wants none of what the neighbouring package already holds.
0016 applied it before the vocabulary existed. A confusion matrix takes int labels and double
weights and would work identically for a model that never saw a string, so the metrics could not be
a namespace in the text package โ and the split is cheapest at the moment a package has no users,
because moving a published type is a break for every consumer that referenced it. 0096 applied it
to ordinary least squares against ten hypothesis-test families: a caller who wants a regression
table wants none of them, and a caller running a Kruskal-Wallis wants no QR.
The same test refuses a package three times, which is what keeps it a test rather than a habit:
-
0111measured the generalized linear model against all three criteria and found none distinct โ the same zero external dependencies and the same two edges before and after, a caller who wants more of the same table rather than none of it, no cadence of its own, and anInternal/LeastSquares.csthat both halves call. It ships beside the OLS it was built next to. -
0119put the explained variance inLodestar.Decompositionrather thanLodestar.Preprocessing, because the package's subject was already the decompositions this repository writes by hand โQrDecomposition.Householdertakes a dense span and noCsrMatrixโ and the adjacency argument for the pipeline package would have cost either a second copy of the Jacobi kernel or a whole edge for one member. -
0124kept the Cox model inLodestar.Survival: its caller holds the same censored durations a Kaplan-Meier curve is fitted on, and usually fits the curve first.
A core package carries no external dependency (0076). A satellite carries the one dependency
that is its whole reason to be a package, and is named for it. The interop tier is
Lodestar.Extensions.<Dependency>, and it may take a dependency a core package refused (0089).
0076 turned the tier table into a rule a script can check: OnnxTextEmbedder left
Lodestar.Embeddings for a Lodestar.Onnx named after Microsoft.ML.OnnxRuntime, so
dotnet add package Lodestar.Embeddings no longer restores a native runtime. The netstandard2.0
polyfills and System.Text.Json are not external dependencies for this rule: they are shims for
what is in-box on net10.0, so a package carrying them offers one API at two implementations rather
than a second thing to install.
0089 settled the case where the two halves of the rule pull apart. Lodestar.Decomposition
refused MathNet.Numerics on freshness because it was choosing a dependency to compute with,
and a stale library there is a liability its users inherit without asking.
Lodestar.Extensions.MathNet takes the same dependency because it is choosing one to convert
to: its entire value is that the caller already holds those types, so refusing would protect
nobody. What a satellite may not do is reach for a stale dependency to implement something.
A framework floor is a permission, not an obligation. Lodestar.Onnx,
Lodestar.Extensions.MathNet and Lodestar.Extensions.VectorData all ship the default pair because
their dependencies do. Lodestar.Gpu is the one package that does not: ILGPU publishes no
netstandard2.0 asset, which is an absence upstream rather than a gap a polyfill closes, and does
publish netstandard2.1. 0101 read the first fact and jumped to net10.0 alone; 0103 amended
it, and the shipped truth is net10.0;netstandard2.1 โ the floor ILGPU itself declares, reaching
.NET 8, Mono and Unity, with a tests/Lodestar.Gpu.NetStandard.Tests mirror pinned to 2.1 so the
second assembly is executed rather than merely compiled. Nothing under src/ may depend on
Lodestar.Gpu, so the SIMD path stays the only path a netstandard2.0 caller has, and stays
complete.
An internal layer opens one member at a time, each time a second package names it, because publishing later is always available and unpublishing never is.
0081 kept Lodestar.Stats' numerical layer internal and refused a public Lodestar.Stats.Special
namespace: each member would become a parity promise in its own right, with its own reference page,
its own wiki-map.json row, its own use in the packaging sample and its own corpus at a tolerance a
general-purpose caller would need rather than the one ten tests happen to need. It wrote its own
escape hatch, and 0095 spent it โ four members, Distributions.StudentSf,
Distributions.StudentQuantile, Distributions.FisherSf and QrDecomposition.Householder, for the
OLS table. 0097 and 0098 apply that rule without amending it: Distributions.ChiSquaredSf
for the log-rank test, and Distributions.NormalQuantile for the Kaplan-Meier log-log bound after
the obvious substitute โ a large-df Student quantile โ was measured and floored at about 9e-9,
which the transform pushes past the 1e-9 the corpora are compared at. 0124 is the same rule read
from the other side: the Cox table needed two tails, both already public, so nothing new was
published.
Publishing is also where a member's name is tested. Both quantiles turned out to be inverse survival functions internally, returning the opposite sign to the printed tables; the published members negate, which is the distribution's symmetry rather than a correction.
A function published this way stays in the package that owns it, and its neighbour takes an edge:
the numerical layer did not move to Lodestar.Abstractions (0081, generalised by 0095), and (e)
says why nothing that computes ever does.
A member another Lodestar package publishes is depended on, not copied; where depending would
make a cycle, the member moves to Lodestar.Abstractions instead (0138, 0071).
0071 is the move: CsrMatrix and SparseNorm left Lodestar.Text because a package wanting a
sparse matrix and two products would otherwise have taken the distances, the phonetics, the
stemmers, the tokenizers, the vectorizers, the persistence layer and System.Text.Json with it.
0138 is the precedent for the ordinary case โ RobustScaler(unit_variance=True) divides by a
normal quantile this repository already publishes and tests against scipy, and two copies drift.
0139 applied it the same week for CsrMatrix, and recorded the thing worth recording: a second
edge in the same package inside one lot needed no fresh argument, while a first edge from another
package still earns its own record.
An edge costs four declarations: the EXPECTED entry in tools/check_nuspec_dependencies.py,
CLAUDE.md's table, the .csproj โ a PackageReference on a
published floor, a ProjectReference behind LodestarUseProjectRefs for the developer loop, and
its own ProjectReference in the netstandard mirror, because SetTargetFramework does not cross a
PackageReference โ and the floor itself in src/Directory.Packages.props. Because the floor names
a published version, the depended-on package ships before the consumer's next release, never
after: a two-package lot is two pull requests with a release between them, which is what
Lodestar.Survival waited on twice.
An edge is not a decision, and this record does not list them. The first text did, and counted itself among an edge's declarations, which left it wrong the day the seventeenth edge landed and unable to follow any edge after. A new edge โ a first one from its package or not โ earns a record only when it changes a rule stated here.
Lodestar.Abstractions holds the public types, enums, interfaces and data-transfer objects the
packages declare, and no code: no logic, no validation, no numerical layer. A type belongs there
when every member the compiler did not write is absent โ no hand-written accessor, setter,
constructor body or method, no non-public member, and nothing it names that stays behind โ with
one allowance: a structural Equals/GetHashCode, comparing the type's own collections by value,
together with the shared ValueEquality helper it calls, which the package then compiles. Interface
members may be abstract; none may have a body. The test runs on the IL, not on a reading of the
source, so it is a rule rather than a judgement.
One exception by name: CsrMatrix, SparseNorm, the products 0071 moved with them and the
ElementWise helper that implements those products stay the sparse primitive they are, code
included. CsrMatrix.CreateUnchecked, the factory Lodestar.Text's vectorizers call to skip a
validation their output does not need, is public for the reason the next paragraph gives. No other
type in the package carries logic.
Lodestar.Abstractions grants no InternalsVisibleTo โ to Lodestar.Text or to its own tests.
A package every other one depends on cannot also let one of them read its internals: the grant is
what forced the package to refuse the shared helpers (#440), and an internal a published consumer
was compiled against becomes a contract nothing versions.
A type keeps its namespace and changes its assembly. Lodestar.Stats.NanPolicy is compiled into
Lodestar.Abstractions.dll and is still Lodestar.Stats.NanPolicy; the package it left declares
[assembly: TypeForwardedTo(typeof(NanPolicy))]. An unchanged full name is what lets a forwarder
keep both binary and source compatibility, and it spares the three names two packages declare
apiece โ BinStrategy, MinHashScheme, ArtifactLoadOptions. Only CsrMatrix and SparseNorm
live in the Lodestar.Abstractions namespace.
What stays in its package: a result with no public constructor, whose construction is the
algorithm's; a data type that validates in its setters or its constructor โ OlsOptions through
OptionGuards, until that validation is detached; a type with a computed member, or one that
reaches an internal of its package โ TTestResult.ConfidenceInterval
through Internal.Beta.StudentQuantile, WordPieceVocabulary's
Count; and any type that names one of those.
Three alternatives lost:
-
Flattening into the
Lodestar.Abstractionsnamespace. A changed full name defeats a forwarder, so every moved type would break its callers at source and at run time, and three names would collide. - Keeping the exchange rule. A type's home would depend on whether a second package had named it yet, and it would move the day one did.
- Admitting data types that validate. The package would carry code, and "no code" would stop being something the IL can check.
Merging or renaming the types two packages both declare is left to 1.0.
The default pair is net10.0;netstandard2.0; only the exception is listed.
| package | tier | holds | target frameworks |
|---|---|---|---|
Lodestar.Abstractions |
core | the public data types the packages declare, under their own namespaces, and CsrMatrix, SparseNorm and the dense-block products โ the sparse primitive the others share |
|
Lodestar.Text |
core | distances, phonetics, set similarity, stemmers, tokenizers, sparse vectorizers, persistence, BkTree, keyword extraction, BM25 |
|
Lodestar.Embeddings |
core | sub-word tokenizers, the batch encoding pipeline, pooling, the SIMD kNN EmbeddingIndex, .npy interop |
|
Lodestar.Fuzzy |
core |
fuzz.*, process.extract, blocking deduplication |
|
Lodestar.Metrics |
core | classification, regression, clustering and ranking metrics | |
Lodestar.Conformal |
core | split conformal intervals and prediction sets | |
Lodestar.Decomposition |
core | truncated SVD and NMF over a CsrMatrix, the Householder QR, the variance principal components explain |
|
Lodestar.Cluster |
core | k-means by Lloyd's algorithm over a row-major span | |
Lodestar.Preprocessing |
core | scaling, encoding, imputation and the cross-validation splitters, dense and sparse | |
Lodestar.Stats |
core | the hypothesis tests, and the tail members 0095, 0097 and 0098 published for its neighbours |
|
Lodestar.Stats.Regression |
core | ordinary, weighted and generalized least squares with the inference table, and the GLM 0111 kept here |
|
Lodestar.Stats.TimeSeries |
core | the autocorrelation functions, Ljung-Box, augmented Dickey-Fuller, KPSS, seasonal decomposition | |
Lodestar.Survival |
core | Kaplan-Meier, Nelson-Aalen, the log-rank test and the Cox model 0124 kept here |
|
Lodestar.Onnx |
satellite |
OnnxTextEmbedder, and the reason the tier exists: Microsoft.ML.OnnxRuntime
|
|
Lodestar.Extensions.AI |
interop | the ONNX embedding path behind IEmbeddingGenerator; carries Microsoft.Extensions.AI.Abstractions
|
|
Lodestar.Extensions.MathNet |
interop |
CsrMatrix to and from Math.NET's sparse matrix; carries MathNet.Numerics
|
|
Lodestar.Extensions.VectorData |
interop | an in-process VectorStore with hybrid search; carries Microsoft.Extensions.VectorData.Abstractions
|
|
Lodestar.Gpu |
satellite | ILGPU kernels over device-resident matrices and text; no src/ project may depend on it |
net10.0;netstandard2.1 |
The graph is tools/check_nuspec_dependencies.py's EXPECTED, floors included, since an edge with
the wrong floor is a different edge; CLAUDE.md restates it and tools/check_claude_md_packages.py
holds the two together. Two edges carry a reason worth keeping here, because they are the ones the
rules above were written from: Lodestar.Text โ Lodestar.Abstractions is 0071's move, and
Lodestar.Preprocessing โ Lodestar.Stats is 0138's precedent for depending rather than copying.
Nothing takes an edge into Lodestar.Gpu, as (b) requires.
-
tools/check_nuspec_dependencies.py'sEXPECTEDremains the authority: an unexpected dependency fails as loudly as a missing one, and so does a moved version range.CLAUDE.md's package table and edge count follow it, andtools/check_claude_md_packages.pyfails when they drift. - Counts in the merged records of the first numbering were right on their dates and are not today โ
0076's layout table lists eight packages,0100callsLodestar.Extensions.VectorDatathe fifteenth,0111concludes the repository stays at sixteen. This record states none, for the same reason. -
0101's title and decision sentence โLodestar.Gputargetsnet10.0alone โ is the claim0103amended, and this record states the amended truth.netstandard2.0is still absent upstream and still not offered. -
0016is written throughout in the repository's formerDataNet.*naming; every package it names ships asLodestar.*. Its reasoning is unaffected. - A new edge, and a further member published from
Lodestar.Stats' numerical layer to a named caller, are routine under (c) and (d); either needs a record only if it changes a rule here. - (e) moves 94 types, and every package that declared one takes an edge to
Lodestar.Abstractions, which therefore ships before each of them (#1142). From then on, changing a moved type is a two-package change, which is the price of the rule. - A namespace now spans two assemblies, so whatever reads a namespace's types โ the reference gate, the sample coverage check โ reads both.