Text

Start here

• Quickstart
• From string to vector — bag of words, TF-IDF, hashing
• Indexing strings for fast lookup — Lodestar.Text.Indexing
• Keyword extraction
• Keyword search
• Lodestar.Text
• Performance — Lodestar.Text

Namespaces

• Core — Lodestar.Text — The Lodestar.Text namespace itself holds one type, and it is one every other namespace in the package takes as a parameter: TextElement, which says what counts as one character.
• Distances — Lodestar.Text.Distances — How different are two pieces of text?
• String indexing — Lodestar.Text.Indexing — Comparing a query against every entry in a dictionary is O(n) per lookup, and that cost is paid again for every query.
• Keyword extraction — Lodestar.Text.Keywords — Two ways to pull the words that matter out of a document, and neither reads the document twice to do it — the score comes out of the same graph or the same run table that finds the candidates in the first place.
• Persistence — Lodestar.Text.Persistence — A fitted vectorizer is worth saving: the vocabulary and the document frequencies came from a training corpus, and that corpus may not be around later.
• Phonetic encoding — Lodestar.Text.Phonetics — Smith and Smyth are the same name.
• Keyword search — Lodestar.Text.Search — BM25 over a document-term matrix, and a way to fuse the ranking it produces with another.
• Set similarity — Lodestar.Text.Similarity — How much do two pieces of text have in common, when where it appears does not matter?
• Stemming — Lodestar.Text.Stemming — ran, running and runs are three spellings of one idea, and an index that stores them as three terms cannot match a query that uses the fourth.
• Vectorization — Lodestar.Text.Vectorization — You have a corpus of documents and you need numbers.