Text textrankoptions - CyrilB1531/lodestar GitHub Wiki

Development build. This page describes main, not a released package. The latest published Lodestar.Text is 0.7.0 — read its documentation.

Home › Text › Keyword extraction

TextRankOptions

What TextRank is built with.

public sealed record TextRankOptions

Properties — StopWords (default null, which takes StopWords.English) is dropped before the graph is built. Window (default 2) is how many tokens share a co-occurrence window; 2 pairs adjacent tokens only. Damping (default 0.85) is the random-surfer damping of the reference implementation. Tolerance (default 1e-12) is this implementation's own convergence bound — summa solves the eigenproblem outright and has no tolerance to expose; it must be finite and non-negative. MaxIterations (default 1_000) is how many power-iteration steps Extract runs before giving up rather than return a half-ranked vector. Ratio (default 0.2) is what proportion of ranked words to keep, ignored when Words is set. Words (default null) overrides Ratio with an exact count, non-negative when set. TokenPattern (default \b\w+\b) is what counts as a word, read as Python's re reads it and taken as re.findall takes it — see CountVectorizerOptions.

Example — Words overriding Ratio on the same document.

using Lodestar.Text.Keywords;

string doc =
    "Compatibility of systems of linear constraints over the set of natural numbers. " +
    "Criteria of compatibility of a system of linear Diophantine equations.";

var byRatio = new TextRank(new TextRankOptions { Ratio = 0.2 }).Extract(doc);
var byCount = new TextRank(new TextRankOptions { Ratio = 0.2, Words = 5 }).Extract(doc);

bool overridden = byRatio.Count != byCount.Count;  // => True

Remarks — Tolerance and MaxIterations have no counterpart in summa, which calls scipy.linalg.eig rather than iterating: this implementation's own power iteration is what makes Tolerance and MaxIterations parameters at all, and their defaults are tight enough that TextRankOracleTests passes at the 1e-9 this repository checks every oracle corpus against — replaying the loop by hand against the frozen corpus measures the agreement itself well past that, under 4.48e-13 on the loosest case. Tolerance's own 1e-12 is the loop's convergence delta, not that distance — the two are different quantities.

StopWords is taken and exposed as given, not copied: writing to the caller's collection changes what these options equal, and every holder of it, a with copy included (#1305).

Applies to — net10.0, netstandard2.0.

See also — TextRank, KeywordMatch, the Python equivalence table.

Members

member what it does
TextRankOptions.Equals Value equality, the stop words as a set.
TextRankOptions.GetHashCode A hash consistent with it.