Development build. This page describes main, not a released package. The latest published Lodestar.Text is 0.7.0 — read its documentation.

Home › Text › Vectorization

HashingVectorizer

Counts into a fixed number of columns, learning nothing — the equivalent of sklearn.feature_extraction.text.HashingVectorizer.

Each term is hashed to a column. There is no vocabulary, so there is no Fit, no memory that grows with the corpus, and no GetFeatureNames: nothing was kept that could name a column.

public sealed class HashingVectorizer

Constructor — HashingVectorizer(HashingVectorizerOptions? options = null), whose defaults are scikit-learn's. It throws ArgumentOutOfRangeException when NumFeatures is below 1, and ArgumentException when Count.NgramRange descends, the one range scikit-learn refuses too, or when Count.Analyzer is not an AnalyzerKind member. The token pattern is checked after NumFeatures and the n-gram range, where 0.7.0 compiled it, whatever the analyzer: a null one throws ArgumentNullException with the parameter named pattern, and one .NET refuses as written its RegexParseException, a leading (?u) included, ahead of the analyzer check — a plain ArgumentException on a runtime without that type, as 0.7.0 raised there (#1645, #1656).

Properties — NumFeatures is how many columns the matrix has.

Example — no fitting, and a width chosen rather than discovered.

using Lodestar.Abstractions;
using Lodestar.Text.Vectorization;

var hv = new HashingVectorizer(new HashingVectorizerOptions { NumFeatures = 16 });
CsrMatrix hashed = hv.Transform(["the cat eats", "the dog eats", "the cat and the dog"]);

int columns = hashed.ColumnCount;  // => 16
double rowLength = hashed.RowL2Norm(0);  // => 1

Remarks — the trade is stateless-ness for names and for collisions. Choose this when the corpus is a stream too large to pass over twice, when documents arrive one at a time and the vocabulary would grow without bound, or when several machines must produce compatible vectors without sharing a fitted model — hashing is deterministic, so they will.

Do not choose it when you will need to explain a vector. Column 9 means "whatever hashed to 9", possibly two unrelated terms at once, and there is no way back.

AlternateSign is what keeps collisions from simply accumulating; see HashingVectorizerOptions.

Applies to — net10.0, netstandard2.0.

See also — HashingVectorizerOptions, CountVectorizer, CsrMatrix, the vectorization guide.

Members

Member	What it does
HashingVectorizer.FitTransform	The same as Transform; there is nothing to fit.
HashingVectorizer.Load	Read the options back.
HashingVectorizer.LoadAsync	The same, without blocking.
HashingVectorizer.Save	Write the options out.
HashingVectorizer.SaveAsync	The same, without blocking.
HashingVectorizer.Transform	Hash a corpus into the fixed columns.