Development build. This page describes main, not a released package. The latest published Lodestar.Extensions.VectorData is 0.1.0 — read its documentation.

Home › ExtensionsVectorData › Vector store

LodestarVectorStoreOptions

How a collection builds its keyword half, and the offset reciprocal-rank fusion uses.

public sealed class LodestarVectorStoreOptions

Properties — all three are init-only, and each defaults to what the member it configures does on its own.

• Vectorizer is the CountVectorizerOptions the full-text values are tokenized and counted with, and the query keywords with them. null, the default, takes CountVectorizer's defaults — scikit-learn's, single-letter words dropped.
• Bm25 is the Bm25Options the keyword index scores with. null, the default, takes rank_bm25's k1 = 1.5, b = 0.75.
• RankFusionK is the k in 1 / (k + rank) that LodestarVectorStoreCollection.HybridSearchAsync fuses with, 60 by default — RankFusion.DefaultK. Setting it to zero or below throws ArgumentOutOfRangeException.

Example — the defaults, and a store whose keyword half drops English stop words.

using Lodestar.Extensions.VectorData;
using Lodestar.Text.Vectorization;

var defaults = new LodestarVectorStoreOptions();
int k = defaults.RankFusionK;  // => 60

var english = new LodestarVectorStoreOptions
{
    Vectorizer = new CountVectorizerOptions { StopWords = StopWords.English },
    RankFusionK = 20,
};
using var store = new LodestarVectorStore(english);

int tighter = english.RankFusionK;  // => 20

Remarks — RankFusionK decides how much one ranking's first place is worth against agreement between the two. At k = 1, a record first in one ranking and absent from the other (1/2) outscores a record fourth in both (2/5). At k = 60 the same record (1/61) loses to one ranked anywhere up to 61st in both, and ties one ranked 62nd in both. Sixty is the value Cormack et al. (2009) used, and the one RankFusion.Rrf takes by default.

The options are checked when first used, by the first hybrid search over a non-empty collection, rather than at construction: an out-of-range Bm25 or Vectorizer setting, or a MaxDf below MinDf over the records held, fails that search, and a vector search never reads them. Every property is init-only, and a stop-word collection handed in is not to be mutated afterwards: texts already tokenized would not follow it.

A store passes one options object to every collection it creates. A collection that needs different options is constructed directly, with its own.

Applies to — net10.0, netstandard2.0.

See also — LodestarVectorStore, LodestarVectorStoreCollection, Bm25Index.