Text simhash hammingdistance - CyrilB1531/lodestar GitHub Wiki

Development build. This page describes main, not a released package. The latest published Lodestar.Text is 0.6.0 — read its documentation.

HomeTextSet similarity

SimHash.HammingDistance

How many bits two fingerprints differ in.

public static int HammingDistance(ulong left, ulong right)

Parametersleft and right are two fingerprints.

Returnsint, a count in [0, 64]. Zero means the two documents fingerprint alike.

Example — the shape a caller writes.

using Lodestar.Text.Similarity;

ulong same = SimHash.Fingerprint(["alpha", "beta"]);

int apart = SimHash.HammingDistance(same, same);  // => 0

Remarks — the usual near-duplicate threshold is three bits or fewer out of 64, which is the figure the published work uses for web pages. It is not a universal constant: it depends on how many tokens a document holds, and a short document's fingerprint moves further per token than a long one's.

Zero means the fingerprints agree, not that the documents do. Sixty-four bits over a large corpus will collide, which is why a fingerprint match is a candidate to verify rather than an answer — the same relationship LshIndex has with MinHash.Jaccard.

Applies to — net10.0, netstandard2.0.

See alsoSimHash.Fingerprint.