Hash_Table - peregrineshahin/ChessProgrammingWiki GitHub Wiki

title: Hash Table

Home * Programming * Data * Hash Table

[ A small phone book as a hash table [1] A Hash Table, or a Hash Map, is a data structure that associates identifiers or keys (names, chess positions) with values (i. e. phone number, score of a position). A hash function is used for turning the key into a relatively small integer, the hash, that serves as an index into an array.

In a well-dimensioned hash table, the average cost for each lookup is independent of the number of elements stored in the table. In general programming as well in computer chess, hash tables often serve as cache for once calculated values, to save relative expensive computations over and over again.

Perfect Hashing

[ A perfect hash function [2] If all of the keys are known at compile or initialization time and their cardinality is reasonable small, a perfect hash table can be created, in which there will be no collisions, since each key has an unique index. Opposed to Minimal Perfect Hashing, the lookup array contains either gaps or multiple entries.

Applications in computer chess are hashing of masked occupied bitboards, to map for instance attack sets of sliding pieces (rooks, bishops) on a particular square, or pawn shield stuff. Another application, despite a reversible hash function, is a precomputed table indexed by some material key, likely an incremental updated dot-product of all piece-type counts in some fixed order, by a vector of cumulated maximum count products of pieces ordered below, usually ignoring unusual material configurations due to promotions [3] . Persistent, or on the fly generated Endgame Tablebases and Bitbases containing perfect knowledge by retrograde analyses of certain material constellations with a few pieces in late endings might be considered as a kind of perfect hashing as well.

Minimal Perfect Hashing

[ Minimal perfect hash function [4] If the hash array has no gaps and unique, distinct values, so called Minimal Perfect Hashing is applied, like in following bitboard hashing for bitscan purpose or determining pre-calculated move lists by move target sets of knights and king.

Search Tables

The classical hash table implementation in computer chess are the transposition tables, indexed by Zobrist- or BCH keys of the position, modulo number of entries inside the table, to store search results, evaluation, pawn structure related stuff and repetitions.

Class Libraries

QMap Class Reference, C++ from Qt On-line Reference Documentation
CMap Class, C++ from MFC Library Reference
Interface Map from the Java package java.util
UserDict - Class wrapper for dictionary objects from Python Library Reference
defaultdict objects from Python Library Reference

Publications

1968

Ward Douglas Maurer (1968). An Improved Hash Code for Scatter Storage. Communications of the ACM, Vol. 11, No. 1
Elwyn Berlekamp (1968). Algebraic Coding Theory, New York: McGraw-Hill. Revised ed., Aegean Park Press, (1984), ISBN 0894120638. amazon
F R A Hopgood (1968). A Solution to the table overflow problem for Hash Tables. Literature: Reports hosted by Atlas Computer Laboratory

1970 ...

Albert Zobrist (1970). A New Hashing Method with Application for Game Playing. Technical Report #88, Computer Science Department, The University of Wisconsin, Madison, WI, USA. Reprinted (1990) in ICCA Journal, Vol. 13, No. 2, pdf
Burton H. Bloom (1970). Space/time trade-offs in hash coding with allowable errors. Comm. of the ACM, Vol. 13, No. 7, pdf [5]
F R A Hopgood, J. Davenport (1972). The quadratic hash method when the table size is a power of 2. Literature: Reports hosted by Atlas Computer Laboratory
F R A Hopgood (1974). Computer Animation used as a Tool in Teaching Computer Science. Literature: Reports hosted by Atlas Computer Laboratory
Ole Amble, Donald Knuth (1974). Ordered Hash Tables. The Computer Journal, Vol. 17
Ward Douglas Maurer, Ted G. Lewis (1975). Hash Table Methods. ACM Computing Surveys, Vol. 7, No. 1
J. Lawrence Carter, Mark N. Wegman (1977). Universal classes of hash functions. STOC '77

1980 ...

Harry Nelson (1985). Hash Tables in Cray Blitz. ICCA Journal, Vol. 8, No. 1 » Cray Blitz
Tony Warnock, Burton Wendroff (1988). Search Tables in Computer Chess. ICCA Journal, Vol. 11, No. 1
Ivan Damgård (1989). A Design Principle for Hash Functions. CRYPTO '89
Ralph C. Merkle (1989). One Way Hash Functions and DES. CRYPTO '89 [6]

1990 ...

Zbigniew J. Czech, George Havas, Bohdan S. Majewski (1992). An Optimal Algorithm for Generating Minimal Perfect Hash Functions. Information Processing Letters, Vol. 43, pp. 257–264. pdf
Warren D. Smith (1992). Hash functions for Binary and Ternary Words. NEC Research Institute, ps
Martin Dietzfelbinger, Anna Karlin, Kurt Mehlhorn, Friedhelm Meyer auf der Heide, Hans Rohnert, Robert E. Tarjan (1994). Dynamic Perfect Hashing: Upper and Lower Bounds. SIAM Journal on Computing, Vol. 23, Nr. 4
Zbigniew J. Czech, George Havas, Bohdan S. Majewski (1997). Perfect Hashing. Theoretical Computer Science, Vol. 182, Nos. 1-2, pp. 1-143
Bob Jenkins (1997). Hash functions. Dr. Dobb's Journal, September 1997 [7]
Donald E. Knuth (1998). The Art of Computer Programming. Volume 3 - Sorting and Searching, 6.4 about universal hashing, (second edition), amazom
Dennis Breuker (1998). Memory versus Search in Games. Ph.D. Thesis, Universiteit Maastricht, The Netherlands. ISBN 90-9012006-8.
Charles E. Leiserson, Harald Prokop, Keith H. Randall (1998). Using de Bruijn Sequences to Index a 1 in a Computer Word, pdf » BitScan

2000 ...

Robert Hyatt, Tim Mann (2002). A lock-less transposition table implementation for parallel search chess engines. ICGA Journal, Vol. 25, No. 1 » Shared Hash Table - Lock-less
Robert Hyatt, Anthony Cozzie (2005). The Effect of Hash Signature Collisions in a Chess Program. ICGA Journal, Vol. 28, No. 3
Dimitris Fotakis, Rasmus Pagh, Peter Sanders, Paul Spirakis (2005). Space Efficient Hash Tables with Worst Case Constant Access Time. Theory of Computing Systems, Vol. 38, No. 2
Sam Tannous (2007). Avoiding Rotated Bitboards with Direct Lookup. ICGA Journal, Vol. 30, No. 2, pdf » Hashing Dictionaries
Kumar Chellapilla, Anton Mityagin, Denis Xavier Charles (2007). GigaHash: scalable minimal perfect hashing for billions of urls. WWW 2007
Trevor Fenner, Mark Levene (2008). Move Generation with Perfect Hashing Functions. ICGA Journal, Vol. 31, No. 1, pdf » Congruent Modulo Bitboards

2010 ...

Maurizio Monge (2010). On perfect hashing of numbers with sparse digit representation via multiplication by a constant. arXiv:1003.3196 » Magic Bitboards
Jiao Wang, Si-Zhong Li, Xin-He Xu (2010). A Minors Hash Table in Chinese-Chess Programs. ICGA Journal, Vol. 33, No. 1
Mihai Pătrașcu, Mikkel Thorup (2011). The Power of Simple Tabulation Hashing. arXiv:1011.5200v2
Dan Anthony Feliciano Alcantara (2011). Effcient Hash Tables on the GPU. Ph.D. thesis, University of California, Davis, pdf » GPU
John Tromp (2014). Cuckoo Cycle: a memory-hard proof-of-work system. IACR Cryptology ePrint Archive [8] [9]
David Eppstein, Michael T. Goodrich, Michael Mitzenmacher, Paweł Pszona (2014). Wear Minimization for Cuckoo Hashing: How Not to Throw a Lot of Eggs into One Basket. arXiv:1404.0286
Thomas Dybdahl Ahle, Martin Aumüller, Rasmus Pagh (2016). Parameter-free Locality Sensitive Hashing for Spherical Range Reporting. arXiv:1605.02673 [10]
Tobias Maier, Peter Sanders, Roman Dementiev (2016). Concurrent Hash Tables: Fast and General?(!). arXiv:1601.04017
Thomas Dybdahl Ahle (2017). Optimal Set Similarity Data-structures Without False Negatives. Master thesis, University of Copenhagen, pdf [11]
Tobias Maier, Peter Sanders (2017). Dynamic Space Efficient Hashing. arXiv:1705.00997
Peter Sanders (2018). Hashing with Linear Probing and Referential Integrity. arXiv:1808.04602

2020 ...

Niklas Fiekas (2020). The Hashtable Packing Problem. (Draft)
Thomas Dybdahl Ahle, Jakob Tejs Bæk Knudsen, Mikkel Thorup (2020). The Power of Hashing with Mersenne Primes. arXiv:2008.08654 [12]

Forum Posts

1998 ...

Fast hash algorithm by John Scalo, CCC, January 08, 1998 » Zobrist Hashing
Fast hash key method - Revisited! by John Scalo, CCC, January 14, 1998
Hash tables and data-cache, some programmer stuff... by Ed Schröder, CCC, January 17, 1998

2000 ...

A fast hash -- assuming you are not planning to do incremental updates by Dann Corbit, CCC, June 15, 2001
Non power of two hash table sizes by Alvaro Jose Povoa Cardoso, CCC, February 18, 2002

2005 ...

Hash table and O(1) by Dann Corbit, CCC, February 06, 2006
What is the Difference between Mainhash, Evalhash, Pawnhash? by Thomas Wallendik, Hiarcs Forum, October 13, 2007
Cache pollution when reading/writing hash table by Marco Costalba, CCC, August 09, 2009

2010 ...

Cache-friendier material index by Harm Geert Muller, CCC, March 31, 2010
how to measure frequency of hash collisions by Daniel Shawul, CCC, June 16, 2012 » Key Collisions, Transposition Table
Hashing based on move lists by Matthew R. Brades, CCC, June 30, 2013

2015 ...

Hash cache by Harm Geert Muller, CCC, October 12, 2015 » Transposition Table
On-the fly hash key generation? by Evert Glebbeek, CCC, January 12, 2016
Crafty's four hash tables by Louis Zulli, CCC, January 17, 2016 » Crafty
Hashed repetition table by J. Wesley Cleveland, CCC, September 04, 2016 » Repetition Hash Table
Hashing a quadboard from scratch by Edoardo Manino, CCC, November 23, 2016 » Quad-Bitboards
Magic end-game material hash? by Harm Geert Muller, CCC, November 30, 2017
hashing in chess4j by James Swafford, CCC, December 30, 2017 » chess4j, Transposition Table

2020 ...

hash collisions by Jon Dart, CCC, January 28, 2020 » Key Collisions
Dense board representation as hash code by koedem, CCC, February 01, 2020
Hashtable packing (e.g. to optimize magic bitboards) is strongly NP-complete by Niklas Fiekas, CCC, February 13, 2020 » Magic Bitboards [13]
Zobrist key independence by Harm Geert Muller, CCC, February 17, 2020 » Zobrist Hashing

External Links

Secure Hash Algorithm from Wikipedia Merkle–Damgård construction from Wikipedia

Koorde based on Chord and De Bruijn Sequence

Transposition-driven scheduling - Wikipedia
Hashlife from Wikipedia by Bill Gosper [14]
Integer Hash Function by Thomas Wang
Hash functions by Paul Hsieh
Hash Functions and Block Ciphers by Bob Jenkins
A Hash Function for Hash Table Lookup by Bob Jenkins
Perfect Hashing by Bob Jenkins
Jenkins hash function from Wikipedia
Fowler–Noll–Vo hash function from Wikipedia
General Purpose Hash Function Algorithms by Arash Partow
Hash Functions
7. Wahl einer geeigneten Hash-Funktion from Guugelhupf - Design und Implementation (German)
Association P.C. - Soft Time in a Life Machine, Berliner Jazztage, November 7, 1971, NDR-RB-SFB 3 Broadcast, YouTube Video

Pierre Courbois, Jasper van 't Hof, Toto Blanke, Sigi Busch

References

↑ Hash function from Wikipedia
↑ Hash function from Wikipedia
↑ Cache-friendier material index by Harm Geert Muller, CCC, March 31, 2010
↑ Hash function from Wikipedia
↑ Bloom filter from Wikipedia
↑ Merkle–Damgård construction from Wikipedia
↑ A Hash Function for Hash Table Lookup by Bob Jenkins, updated Dr. Dobb's article
↑ Cuckoo Cycle: a new memory-hard proof-of-work system by John Tromp, Bitcoin Forum, January 08, 2014
↑ Cuckoo hashing from Wikipedia
↑ Locality-sensitive hashing from Wikipedia
↑ MinHash from Wikipedia
↑ Mersenne prime from Wikipedia
↑ Strong NP-completeness from Wikipedia
↑ Gosper's Algorithm (Hashlife) explained

Up one Level