Text 0.5.0 rake - CyrilB1531/lodestar GitHub Wiki

Lodestar.Text 0.5.0. This page is frozen at that release. Read the current documentation for what main says now. A link to a decision or a migration page follows main, and leaves the archive.

Rake

Rapid Automatic Keyword Extraction: candidates are the runs between stop words, scored by summing a per-word score over the run.

public sealed class Rake

Example — the paper's own worked sentence, with a small stop-word list standing in for StopWords.English. Two candidates tie at the top score, and the tie breaks by phrase descending, rake-nltk's own rule — "natural numbers" before "linear constraints", not the order either one occurs in the source.

using Lodestar.Text.Keywords;

string[] stop = ["of", "the", "over", "a", "and", "are", "for", "all", "to", "in", "is", "this", "that"];
var rake = new Rake(new RakeOptions { StopWords = stop });

var hits = rake.Extract("Compatibility of systems of linear constraints over the set of natural numbers.");
string top = hits[0].Phrase;   // => natural numbers
double score = hits[0].Score;  // => 4

Remarks — the constructor is Rake(RakeOptions? options = null); null takes every default, which is StopWords.English for the stop-word list. It throws ArgumentOutOfRangeException when options.MinLength is below 1, and ArgumentException when options.MaxLength is below options.MinLength — a range no candidate could ever satisfy.

The co-occurrence tables — degree and frequency — are built once, over every surviving run, before any candidate is scored. That ordering is what makes IncludeRepeatedPhrases = false change a score and not only the output: dropping the duplicate before the tables are built removes its contribution to degree and frequency too, not merely its second row in the result (decision 0077).

For a graph-ranked alternative that looks at co-occurrence beyond a single run, see TextRank.

Applies to — net10.0, netstandard2.0.

See alsoRakeOptions, RakeMetric, KeywordMatch, TextRank, the keyword extraction guide.

Members

Member What it does
Rake.Extract Extracts the ranked candidates of one document.