Onnx 0.1.0 inference - CyrilB1531/lodestar GitHub Wiki

Lodestar.Onnx 0.1.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

ONNX inference — Lodestar.Onnx

One type, OnnxTextEmbedder: it runs a sentence-transformer model and gives you a vector per text. It is the only place in Lodestar where a model file is required, and the only place ONNX Runtime is referenced — that dependency is what this package is for. A caller who tokenizes, pools or searches without inferring never restores a native runtime.

Why every example here is unexecuted

Weights are never committed to this repository. A running example would need a model of tens of megabytes, and decisions/0003 rules that out; tools/fetch_*.py pulls vocabularies against a pinned SHA-256 when they are needed, and weights are not among them.

So the fences on these pages compile against the packed package and are marked docs-run: skip, which is what that marker is for. The same exclusion is declared in the packaging sample, where OnnxTextEmbedder is one of its two documented exclusions.

Where the vectors come from, and where they go

This package produces vectors, and does nothing else. Turning text into the token ids it wants is Lodestar.Embeddings.Tokenization; reducing a sequence of vectors to one is Lodestar.Embeddings.Pooling — which this type also does internally; searching a set of them is Lodestar.Embeddings.Search. This page is the middle step of four, and the only one that needs a file from outside, which is why it is the only step that ships apart.

Types

Type What it is
OnnxTextEmbedder Runs an ONNX sentence-transformer and returns vectors.

See also