Computer Chess Club Archives




Subject: Re: Hash Collisions

Author: Robert Hyatt

Date: 08:14:14 02/22/99

Go up one level in this thread

On February 22, 1999 at 02:30:37, Don Dailey wrote:

>On February 21, 1999 at 15:58:16, Robert Hyatt wrote:
>>On February 21, 1999 at 13:23:24, William Bryant wrote:
>>>On February 21, 1999 at 10:21:50, Robert Hyatt wrote:
>>>>On February 21, 1999 at 01:14:37, KarinsDad wrote:
>>>>>I posted this the other day, but it may have been obscured (or not if nobody has
>>>>>any information on it).
>>>>>I have been wondering if changing the Zobrist hash from a set of random number
>>>>>to a set of non-random and very specific numbers could result in a more even
>>>>>distribution in the hash table. Has anyone done any work in this area?
>>>>Burt Wendroff and Tony Warnock published a paper in the JICCA a few years
>>>>ago discussing this.  The main issue is to find the hamming distances between
>>>>_all_ the random numbers, and maximize all of them.  Random numbers work fine
>>>>if they have been tested (I ran all of mine thru a hamming distance analysis
>>>>when I first started).
>>>>As far as 'distribution' this is done by the low order N bits
>>>>(N=log2(hash_size)) of the hash signature.  You _could_ do the hamming analysis
>>>>on just those bits as well.
>>>I check all my random numbers for uniqueness, ie no duplicates,and use
>>>0xFFFFFFFFFFFFFFFF for the side to move (compliment the side to move),
>>>but would appreciate information on 'hamming distance analysis' in not to
>>>complicated to post or send via email.
>>>Thank you,
>>"hamming distance" is simply the number of bits two numbers differ in.  To
>>compute the hamming distance for a and b, you just compute popcnt(a^b).  Two
>>numbers can differ in all bits.  But more than two can not...  I shoot for
>>no worse than 32 bits that are the same in two random numbers...  Assuming
>>64 bit values...
>Are you sure NO two are closer (in hamming distance) than 32 bits?  How
>many numbers are in your table?    Are you talking about the worst
>case or is it some kind of average?

I don't do this in Crafty.  I do (did) in Cray Blitz, because the trees we
searched were far larger.  And what I did there was to generate the random
numbers separately, and each new number had to pass the test against all the
previous numbers before it was accepted. And yes, this was time-consuming so
we made these numbers 'constant'.  And no, I am not sure that all the numbers
were separated by a hamming distance of 32...  that goes back in time far enough
to make it very fuzzy.

However, I don't have that many numbers (1024 you said?) as I only used 12*64
since we didn't use any sort of 'boundary squares' in cray blitz.  Ditto for
Crafty where I also use 12*64.  I have been meaning to go grab that table of
numbers and 'steal' it for crafty, but I haven't yet, because it is in the
syntax of 'fortran'.

>I have a little piece of code I wrote that generates 64 bit random
>numbers one at a time and tests each one against all the ones
>previously generated.  If the new number is closer than my specified
>hamming distance, I regenerate that number until it is.   To get
>even 24 bit distances between any two I've had to generate almost
>half a million numbers.   I have 1024 numbers in my table.
>I don't believe your random number generator is returning numbers
>this good.  Maybe you can precompute them this good, I don't know.
>I'm trying right now with 32 bits but it's going awfully slow.

I use the Numerical Recipes RNG code.  But you are right, it won't produce
such good hamming distances quickly.  I wouldn't be surprised if it takes
4 billion numbers to get decent random numbers...

I'll try to figure out a way to post my values here if you are interested.
64 * 12 isn't horribly big, but when you display them as 64 bit values, they
are big. The main issue is how to initialize such an array in a portable
way...  IE (BITBOARD ran[12][64] = {{1,2,3},{4,5,6}};  The numbers 1,2,3
default to 'int' which is bad.  I don't know what to do for all compilers
(IE 123ULL or (BITBOARD) 123)

>So my code is either wrong or I'm not understanding your method.
>Can you either clarify or send me your table to analyze?
>- Don
>- Don

This page took 0.03 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.