Embeddings 0.7.0 specialtokentemplate equals - CyrilB1531/lodestar GitHub Wiki
Lodestar.Embeddings 0.7.0. This page is frozen at that release. Read the current documentation for what
mainsays now. A link to a decision or a migration page followsmain, and leaves the archive.
SpecialTokenTemplate.Equals
Value equality over the prefix, the suffix and the pad token.
public bool Equals(SpecialTokenTemplate other)
Parameters — other is the template to compare against.
Returns — bool, true when both wrap a sequence identically.
Example — the ready-made templates differ.
using Lodestar.Embeddings.Tokenization;
bool same = SpecialTokenTemplate.Bert.Equals(SpecialTokenTemplate.None); // => False
Remarks — the token lists are compared element by element rather than by reference, which
a record's synthesised equality would do and which would report two identical templates as
different.
Comparing templates is how to check that a saved encoder and a model still agree — the kind of check worth doing once at start-up rather than discovering through bad vectors.
Applies to — net10.0, netstandard2.0.
See also — SpecialTokenTemplate.