Text simhash hammingdistance - CyrilB1531/lodestar GitHub Wiki
Development build. This page describes
main, not a released package. The latest published Lodestar.Text is 0.6.0 — read its documentation.
Home › Text › Set similarity
SimHash.HammingDistance
How many bits two fingerprints differ in.
public static int HammingDistance(ulong left, ulong right)
Parameters — left and right are two fingerprints.
Returns — int, a count in [0, 64]. Zero means the two documents fingerprint alike.
Example — the shape a caller writes.
using Lodestar.Text.Similarity;
ulong same = SimHash.Fingerprint(["alpha", "beta"]);
int apart = SimHash.HammingDistance(same, same); // => 0
Remarks — the usual near-duplicate threshold is three bits or fewer out of 64, which is the figure the published work uses for web pages. It is not a universal constant: it depends on how many tokens a document holds, and a short document's fingerprint moves further per token than a long one's.
Zero means the fingerprints agree, not that the documents do. Sixty-four bits over a large
corpus will collide, which is why a fingerprint match is a candidate to verify rather than an
answer — the same relationship LshIndex has with
MinHash.Jaccard.
Applies to — net10.0, netstandard2.0.
See also — SimHash.Fingerprint.