migration - CyrilB1531/lodestar GitHub Wiki

From Python to .NET β€” migration inventory

This page is Lodestar's migration hub. It answers a simple question: "I do this in Python, what do I do in C#?"

The project's guiding principle (see the rationale) is honest: we don't rewrite Python's data-science ecosystem. Most of it already exists in .NET, and Python's dense linear algebra relies on Fortran BLAS/LAPACK kernels there's no point reimplementing. We use what exists, and only write native code where .NET has no maintained equivalent at the reference's parity.

Seventeen packages in, that gap has a shape. It is almost never the computation β€” .NET ships those β€” and almost always the apparatus around it: the tokenizer loader and not its encoder, the regression's inference table and not its coefficients, the time-series diagnostics and not the forecast, sparse decomposition and not dense. Each of those is a decision record with a reading behind it, linked from the rows below.

The four columns

Python Role .NET recommendation Verdict
PyTorch tensors, autograd, training, GPU TorchSharp (= libtorch); ONNX Runtime for inference only βœ… Use
matplotlib plotting ScottPlot, Plotly.NET, OxyPlot βœ… Use
NumPy N-dim arrays, dense algebra Math.NET Numerics (+ native MKL/OpenBLAS provider); System.Numerics.Tensors βœ… Use
scikit-learn classical ML, pipelines, metrics ML.NET; SharpLearning βœ… Use except text vectorization β†’ Lodestar.Text, classification metrics β†’ Lodestar.Metrics, and the two sparse decompositions (TruncatedSVD, NMF) β†’ Lodestar.Decomposition
MAPIE conformal prediction: intervals and prediction sets with a coverage guarantee none β€” no C# implementation exists πŸ”΄ Write β€” split conformal is Lodestar.Conformal
pandas DataFrame, groupby, IO Microsoft.Data.Analysis; Deedle 🟑 Use (rougher)
statsmodels econometric regression, time series, tests Math.NET (basics) β€” not Accord.NET, see below; Microsoft.ML.TimeSeries for forecasting πŸ”΄ Write β€” the tests and the OLS table ship as Lodestar.Stats and Lodestar.Stats.Regression; forecasting delegates; the GLM table ships beside the OLS one, and the time-series diagnostics ship as Lodestar.Stats.TimeSeries
scipy.stats hypothesis tests, distributions, tails Math.NET Numerics for the distributions and their tails πŸ”΄ Write β€” ten test families at scipy parity ship as Lodestar.Stats. Math.NET has the distributions and no test battery over them, so the gap is the test and not the tail; decision 0002 admitted scipy as a permissive reference and 0003 says which four tail members this publishes for its neighbours
lifelines survival analysis: Kaplan-Meier, Nelson-Aalen, log-rank none β€” the largest void #442 surveyed πŸ”΄ Write β€” right-censored estimators and the log-rank test ship as Lodestar.Survival, at lifelines parity. scikit-survival is the nearest reference in any language and is refused on its licence, not its capability β€” GPL-3.0-or-later, which decision 0002 excludes (decision 0002)
rank_bm25 beside a vector index hybrid keyword-and-vector retrieval, fused in process Microsoft.Extensions.VectorData and its connectors. Read on 2026-09-13 at CommunityToolkit/AI 215a5bad: six of the ten connectors implement hybrid search, and all six are clients of a server; the two in-process ones, InMemory and SqliteVec, implement none πŸ”΄ Write β€” BM25 and reciprocal rank fusion ship in Lodestar.Text, and the in-process hybrid store behind the abstraction as Lodestar.Extensions.VectorData
seaborn tidy statistical viz ScottPlot / Plotly.NET (charts rebuilt) 🟠 Decide β€” statistical presets missing

Legend. βœ… a solid equivalent exists, use it as is. 🟑 an equivalent exists but is less mature than Python; expect some glue. 🟠 the foundation exists but a whole area is missing: a candidate for native code if your usage justifies it. πŸ”΄ nothing exists and this project wrote it. β›” unmaintained β€” do not reach for it, and the row says since when.

Unmaintained, and why that is stated with dates

Pointing a reader at a library nobody has touched in years is worse than saying nothing, so every β›” here carries the last published package and the last commit, both dated. A verdict without them is a claim that rots quietly; with them, a reader can check whether it is still true and this page can be corrected rather than argued about.

library last package last commit reach for
Accord.NET Accord 3.8.0, 19 October 2017 (3.8.2-alpha, November 2017, is the last of any kind) 18 November 2020 Math.NET Numerics for distributions and regression; Lodestar.Metrics for evaluation
Pandas.NET Pandas.NET 0.6.0, 6 November 2023 4 August 2024, deleting its CI workflow Microsoft.Data.Analysis, or Deedle for time series

Accord.NET is the one that matters, because it is still the first result for half of these searches: the package a reader would install predates .NET Core 3.0.

When calling Python is still the right answer

This page recommends .NET libraries because the project's thesis is that most of the ecosystem already exists there. It is not that Python is never the answer. A model that only exists as a Python package, a notebook workflow with a person in it, a one-off analysis β€” for those, calling Python is right, and a migration guide that pretended otherwise would not be trusted on the cases where it is not.

option shape last release
CSnakes source-generates typed C# from Python type hints; embeds CPython in the process CSnakes.Runtime 1.2.1, August 2025
Python.NET dynamic interop, the long-standing option pythonnet 3.1.0, May 2026

Both were active within the last month at the time of writing. What they cost is what Lodestar exists to avoid where it can: a Python runtime to deploy and version alongside the application, no ahead-of-time compilation to a single artifact, and the GIL between your threads and theirs.

What Lodestar writes natively

Text was the first area that justified native code, and it stopped being the only one some time ago: seventeen packages ship now. Each lot below was opened by a reading of what .NET already exports β€” never by an assumption that nothing existed β€” and the ones that found an incumbent delegated instead. That protocol is decision 0004, and it has replaced a gap claim this project made about itself twice, so the readings are worth more than the list.

  1. String distances & similarity β€” Levenshtein, Damerau-Levenshtein, Jaro-Winkler, Jaccard, Ratcliff-Obershelp, phonetics… (done)
  2. Tokenization & sparse vectorization β€” CountVectorizer, TfidfVectorizer (exact sklearn semantics), home-grown CSR matrix. (done)
  3. Embeddings & semantic search β€” ONNX Runtime + sub-word tokenizers. (done) Native for one measured reason: Microsoft.ML.Tokenizers builds every tokenizer from a vocabulary, a merges file or a spiece.model, and cannot read the tokenizer.json that Llama-2 and Mistral v0.1 actually ship. The gap is the loader, not the arithmetic β€” decision 0004 β€” and the encode paths no longer trail it either (performance guide).
  4. Applied fuzzy matching β€” rapidfuzz.fuzz / process equivalents. (done)
  5. Classification metrics β€” sklearn-parity precision, recall, F1, confusion matrix, report and ROC-AUC. (done)
  6. Split conformal prediction β€” MAPIE-parity intervals and prediction sets, with the finite-sample coverage guarantee and the exchangeability assumption it rests on. (done) The survey behind #441 found no C# implementation at all, which is why this one is written rather than delegated; the guarantee's assumption leads its guide rather than closing it.
  7. Sparse truncated SVD and NMF β€” scikit-learn-parity TruncatedSVD and NMF(solver="mu") over a CsrMatrix. (done) Math.NET is still the answer for dense linear algebra and this does not replace it, but its sparse SVD request has been open since 2013, and ML.NET's ProjectToPrincipalComponents centres the data, which densifies the very matrix the sparse representation exists to keep sparse β€” decision 0004 and its guide.
  8. Classical hypothesis tests β€” ten families at scipy parity, plus the four tail members decision 0003 publishes for its neighbours. (done, Lodestar.Stats) Math.NET has the distributions and no battery of tests over them, so the gap is the test and not the tail.
  9. The OLS inference table β€” standard errors, t and p values, confidence intervals, adjusted RΒ², F and its p value, and the VIF, at statsmodels parity. (done, Lodestar.Stats.Regression) The clearest case of the shape above: decision 0003's reading found coefficients everywhere in .NET and inference nowhere, which replaced this project's own sentence that "nobody in .NET does inference".
  10. Right-censored survival β€” Kaplan-Meier, Nelson-Aalen and the log-rank test at lifelines parity. (done, Lodestar.Survival) The largest void #442 surveyed; scikit-survival is refused on its licence rather than its capability (decision 0002).
  11. Feature scaling β€” fitted on arrays, applied to spans, at scikit-learn parity. (done, Lodestar.Preprocessing) ML.NET scales inside a pipeline over an IDataView; this is the same arithmetic with nothing between the caller and a double[].
  12. k-means β€” Lloyd's algorithm over a row-major span, at scikit-learn parity. (done, Lodestar.Cluster) Same reason as the scaling above, and the same measurement.
  13. GPU kernels β€” ILGPU over device-resident matrices and text. (done, Lodestar.Gpu) The one package that does not ship netstandard2.0, and the one whose figures are not measured by CI: a hosted runner has no GPU, so the 5–10Γ— gate is measured on a named machine and published by hand (bench/README.md's GPU gate).
  14. The generalized linear model β€” binomial and Poisson fitted by IRLS, with the same inference table entry 9 describes, at statsmodels parity. (done, Lodestar.Stats.Regression) A GLM exists in .NET three times and none of them is reachable: Accord.Statistics ships the whole stack under LGPL-2.1 and archived in 2017, Microsoft.ML reports coefficient statistics for binary logistic only, and cs-glm installs no assembly at all (decision 0004).
  15. Hybrid retrieval with no database β€” a vector ranking and a BM25 ranking fused by reciprocal rank, behind Microsoft.Extensions.VectorData's IKeywordHybridSearchable. (written, Lodestar.Extensions.VectorData) Every connector of that abstraction that offers hybrid search is a client of a server, and the two that run in process offer none; the arithmetic was already published in Lodestar.Embeddings and Lodestar.Text, so the package is the store around it.

Three more packages carry no lot of their own. Lodestar.Abstractions holds the CsrMatrix the others share; Lodestar.Onnx exists to carry the one dependency that is its whole reason to be a package; Lodestar.Extensions.AI and Lodestar.Extensions.MathNet convert to foreign types, which decision 0003 distinguishes from computing with them.

Per-library guides

Guide Status
NumPy β†’ .NET draft
pandas β†’ .NET draft
scikit-learn β†’ .NET draft
statsmodels β†’ .NET draft
PyTorch β†’ .NET draft
matplotlib β†’ .NET draft
seaborn β†’ .NET draft

There is no scipy.md, lifelines.md or mapie.md, and none is owed. A per-library guide earns its place where the answer is another .NET library and the reader needs the glue and the pitfalls β€” NumPy to Math.NET has real traps, and the page is where they live. For those three the verdict is write: the answer is a Lodestar package, the row above says which and why, and equivalence.md maps each call to its counterpart with the divergences named. A page between them would carry nothing of its own.

The detailed equivalence table (Python call β†’ C# call, behavioral differences, performance notes), filled in as we go, is in equivalence.md.

This document is not legal advice; third-party dependency licenses are recorded in THIRD-PARTY-NOTICES.md.