Embeddings 0.6.0 mmr - CyrilB1531/lodestar GitHub Wiki

Lodestar.Embeddings 0.6.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

Mmr

Maximal Marginal Relevance: greedy selection that trades relevance to a query against redundancy with what is already selected.

public static class Mmr

Example — the same four candidates picked two ways: pure relevance keeps query order, pure diversity reaches for the one orthogonal candidate second.

using Lodestar.Embeddings.Search;

float[] query = [1f, 0f, 0f];
float[][] candidates =
[
    [1.00f, 0.00f, 0.00f],
    [0.80f, 0.60f, 0.00f],
    [0.60f, 0.00f, 0.80f],
    [0.00f, 1.00f, 0.00f],
];

int[] relevanceOnly = Mmr.Select(query, candidates, count: 3, lambda: 1.0);
int[] diverse = Mmr.Select(query, candidates, count: 3, lambda: 0.0);

string relevanceOrder = string.Join(",", relevanceOnly);  // => 0,1,2
string diverseOrder = string.Join(",", diverse);           // => 0,3,2

Remarks — knows nothing about text. The candidates are vectors and the result is their indices, so the same call serves keyword selection (see the keyword extraction guide's KeyBERT-style composition), passage reranking, or any other list a caller wants spread out rather than clustered.

lambda trades the two off: 1 is pure relevance to the query, 0 is pure diversity from what is already picked, and every value between blends the two scores. The first pick is always the most relevant candidate regardless of lambda, because nothing is selected yet to be redundant with.

Applies to — net10.0, netstandard2.0.

See alsoMmr.Select, VectorMath, the search index, the keyword extraction guide.

Members

Member What it does
Mmr.Select Selects up to count candidates.