migration - CyrilB1531/lodestar GitHub Wiki
From Python to .NET β migration inventory
This page is Lodestar's migration hub. It answers a simple question: "I do this in Python, what do I do in C#?"
The project's guiding principle (see the rationale) is honest: we don't rewrite Python's data-science ecosystem. Most of it already exists in .NET, and Python's dense linear algebra relies on Fortran BLAS/LAPACK kernels there's no point reimplementing. We use what exists, and only write native code where .NET has no maintained equivalent at the reference's parity.
Seventeen packages in, that gap has a shape. It is almost never the computation β .NET ships those β and almost always the apparatus around it: the tokenizer loader and not its encoder, the regression's inference table and not its coefficients, the time-series diagnostics and not the forecast, sparse decomposition and not dense. Each of those is a decision record with a reading behind it, linked from the rows below.
The four columns
| Python | Role | .NET recommendation | Verdict |
|---|---|---|---|
| PyTorch | tensors, autograd, training, GPU | TorchSharp (= libtorch); ONNX Runtime for inference only | β Use |
| matplotlib | plotting | ScottPlot, Plotly.NET, OxyPlot | β Use |
| NumPy | N-dim arrays, dense algebra | Math.NET Numerics (+ native MKL/OpenBLAS provider); System.Numerics.Tensors |
β Use |
| scikit-learn | classical ML, pipelines, metrics | ML.NET; SharpLearning | β
Use except text vectorization β Lodestar.Text, classification metrics β Lodestar.Metrics, and the two sparse decompositions (TruncatedSVD, NMF) β Lodestar.Decomposition |
| MAPIE | conformal prediction: intervals and prediction sets with a coverage guarantee | none β no C# implementation exists | π΄ Write β split conformal is Lodestar.Conformal |
| pandas | DataFrame, groupby, IO | Microsoft.Data.Analysis; Deedle |
π‘ Use (rougher) |
| statsmodels | econometric regression, time series, tests | Math.NET (basics) β not Accord.NET, see below; Microsoft.ML.TimeSeries for forecasting |
π΄ Write β the tests and the OLS table ship as Lodestar.Stats and Lodestar.Stats.Regression; forecasting delegates; the GLM table ships beside the OLS one, and the time-series diagnostics ship as Lodestar.Stats.TimeSeries |
| scipy.stats | hypothesis tests, distributions, tails | Math.NET Numerics for the distributions and their tails | π΄ Write β ten test families at scipy parity ship as Lodestar.Stats. Math.NET has the distributions and no test battery over them, so the gap is the test and not the tail; decision 0002 admitted scipy as a permissive reference and 0003 says which four tail members this publishes for its neighbours |
| lifelines | survival analysis: Kaplan-Meier, Nelson-Aalen, log-rank | none β the largest void #442 surveyed | π΄ Write β right-censored estimators and the log-rank test ship as Lodestar.Survival, at lifelines parity. scikit-survival is the nearest reference in any language and is refused on its licence, not its capability β GPL-3.0-or-later, which decision 0002 excludes (decision 0002) |
| rank_bm25 beside a vector index | hybrid keyword-and-vector retrieval, fused in process | Microsoft.Extensions.VectorData and its connectors. Read on 2026-09-13 at CommunityToolkit/AI 215a5bad: six of the ten connectors implement hybrid search, and all six are clients of a server; the two in-process ones, InMemory and SqliteVec, implement none |
π΄ Write β BM25 and reciprocal rank fusion ship in Lodestar.Text, and the in-process hybrid store behind the abstraction as Lodestar.Extensions.VectorData |
| seaborn | tidy statistical viz | ScottPlot / Plotly.NET (charts rebuilt) | π Decide β statistical presets missing |
Legend. β a solid equivalent exists, use it as is. π‘ an equivalent exists but is less mature than Python; expect some glue. π the foundation exists but a whole area is missing: a candidate for native code if your usage justifies it. π΄ nothing exists and this project wrote it. β unmaintained β do not reach for it, and the row says since when.
Unmaintained, and why that is stated with dates
Pointing a reader at a library nobody has touched in years is worse than saying nothing, so every β here carries the last published package and the last commit, both dated. A verdict without them is a claim that rots quietly; with them, a reader can check whether it is still true and this page can be corrected rather than argued about.
| library | last package | last commit | reach for |
|---|---|---|---|
| Accord.NET | Accord 3.8.0, 19 October 2017 (3.8.2-alpha, November 2017, is the last of any kind) |
18 November 2020 | Math.NET Numerics for distributions and regression; Lodestar.Metrics for evaluation |
| Pandas.NET | Pandas.NET 0.6.0, 6 November 2023 |
4 August 2024, deleting its CI workflow | Microsoft.Data.Analysis, or Deedle for time series |
Accord.NET is the one that matters, because it is still the first result for half of these searches: the package a reader would install predates .NET Core 3.0.
When calling Python is still the right answer
This page recommends .NET libraries because the project's thesis is that most of the ecosystem already exists there. It is not that Python is never the answer. A model that only exists as a Python package, a notebook workflow with a person in it, a one-off analysis β for those, calling Python is right, and a migration guide that pretended otherwise would not be trusted on the cases where it is not.
| option | shape | last release |
|---|---|---|
| CSnakes | source-generates typed C# from Python type hints; embeds CPython in the process | CSnakes.Runtime 1.2.1, August 2025 |
| Python.NET | dynamic interop, the long-standing option | pythonnet 3.1.0, May 2026 |
Both were active within the last month at the time of writing. What they cost is
what Lodestar exists to avoid where it can: a Python runtime to deploy and
version alongside the application, no ahead-of-time compilation to a single
artifact, and the GIL between your threads and theirs.
What Lodestar writes natively
Text was the first area that justified native code, and it stopped being the only one some time ago: seventeen packages ship now. Each lot below was opened by a reading of what .NET already exports β never by an assumption that nothing existed β and the ones that found an incumbent delegated instead. That protocol is decision 0004, and it has replaced a gap claim this project made about itself twice, so the readings are worth more than the list.
- String distances & similarity β Levenshtein, Damerau-Levenshtein, Jaro-Winkler, Jaccard, Ratcliff-Obershelp, phoneticsβ¦ (done)
- Tokenization & sparse vectorization β
CountVectorizer,TfidfVectorizer(exact sklearn semantics), home-grown CSR matrix. (done) - Embeddings & semantic search β ONNX Runtime + sub-word tokenizers. (done)
Native for one measured reason:
Microsoft.ML.Tokenizersbuilds every tokenizer from a vocabulary, a merges file or aspiece.model, and cannot read thetokenizer.jsonthat Llama-2 and Mistral v0.1 actually ship. The gap is the loader, not the arithmetic β decision 0004 β and the encode paths no longer trail it either (performance guide). - Applied fuzzy matching β
rapidfuzz.fuzz/processequivalents. (done) - Classification metrics β sklearn-parity precision, recall, F1, confusion matrix, report and ROC-AUC. (done)
- Split conformal prediction β MAPIE-parity intervals and prediction sets, with the finite-sample coverage guarantee and the exchangeability assumption it rests on. (done) The survey behind #441 found no C# implementation at all, which is why this one is written rather than delegated; the guarantee's assumption leads its guide rather than closing it.
- Sparse truncated SVD and NMF β scikit-learn-parity
TruncatedSVDandNMF(solver="mu")over aCsrMatrix. (done) Math.NET is still the answer for dense linear algebra and this does not replace it, but its sparse SVD request has been open since 2013, and ML.NET'sProjectToPrincipalComponentscentres the data, which densifies the very matrix the sparse representation exists to keep sparse β decision 0004 and its guide. - Classical hypothesis tests β ten families at scipy parity, plus the four tail
members decision 0003
publishes for its neighbours. (done,
Lodestar.Stats) Math.NET has the distributions and no battery of tests over them, so the gap is the test and not the tail. - The OLS inference table β standard errors, t and p values, confidence intervals,
adjusted RΒ², F and its p value, and the VIF, at statsmodels parity. (done,
Lodestar.Stats.Regression) The clearest case of the shape above: decision 0003's reading found coefficients everywhere in .NET and inference nowhere, which replaced this project's own sentence that "nobody in .NET does inference". - Right-censored survival β Kaplan-Meier, Nelson-Aalen and the log-rank test at
lifelines parity. (done,
Lodestar.Survival) The largest void #442 surveyed;scikit-survivalis refused on its licence rather than its capability (decision 0002). - Feature scaling β fitted on arrays, applied to spans, at scikit-learn parity.
(done,
Lodestar.Preprocessing) ML.NET scales inside a pipeline over anIDataView; this is the same arithmetic with nothing between the caller and adouble[]. - k-means β Lloyd's algorithm over a row-major span, at scikit-learn parity.
(done,
Lodestar.Cluster) Same reason as the scaling above, and the same measurement. - GPU kernels β ILGPU over device-resident matrices and text. (done,
Lodestar.Gpu) The one package that does not shipnetstandard2.0, and the one whose figures are not measured by CI: a hosted runner has no GPU, so the 5β10Γ gate is measured on a named machine and published by hand (bench/README.md's GPU gate). - The generalized linear model β binomial and Poisson fitted by IRLS, with the same
inference table entry 9 describes, at statsmodels parity. (done,
Lodestar.Stats.Regression) A GLM exists in .NET three times and none of them is reachable:Accord.Statisticsships the whole stack under LGPL-2.1 and archived in 2017,Microsoft.MLreports coefficient statistics for binary logistic only, andcs-glminstalls no assembly at all (decision 0004). - Hybrid retrieval with no database β a vector ranking and a BM25 ranking fused by reciprocal
rank, behind
Microsoft.Extensions.VectorData'sIKeywordHybridSearchable. (written,Lodestar.Extensions.VectorData) Every connector of that abstraction that offers hybrid search is a client of a server, and the two that run in process offer none; the arithmetic was already published inLodestar.EmbeddingsandLodestar.Text, so the package is the store around it.
Three more packages carry no lot of their own. Lodestar.Abstractions holds the
CsrMatrix the others share; Lodestar.Onnx exists to carry the one dependency that is
its whole reason to be a package; Lodestar.Extensions.AI and Lodestar.Extensions.MathNet
convert to foreign types, which
decision 0003
distinguishes from computing with them.
Per-library guides
| Guide | Status |
|---|---|
| NumPy β .NET | draft |
| pandas β .NET | draft |
| scikit-learn β .NET | draft |
| statsmodels β .NET | draft |
| PyTorch β .NET | draft |
| matplotlib β .NET | draft |
| seaborn β .NET | draft |
There is no scipy.md, lifelines.md or mapie.md, and none is owed. A per-library
guide earns its place where the answer is another .NET library and the reader needs the
glue and the pitfalls β NumPy to Math.NET has real traps, and the page is where they live.
For those three the verdict is write: the answer is a Lodestar package, the row above
says which and why, and equivalence.md maps each call to its
counterpart with the divergences named. A page between them would carry nothing of its own.
The detailed equivalence table (Python call β C# call, behavioral differences,
performance notes), filled in as we go, is in equivalence.md.
This document is not legal advice; third-party dependency licenses are recorded in
THIRD-PARTY-NOTICES.md.