Computer Chess Club Archives


Search

Terms

Messages

Subject: Binary Book Creation

Author: Adrien Regimbald

Date: 03:03:43 06/19/00


Hello,

Well, nobody responded when I was asking for this sort of information a while
ago, so I thought that maybe some of you will be interested in seeing my
results, or perhaps you've been looking for this sort of information yourself.

I created the book for Faile at various max book ply (I have set the max ply to
40, so I don't have any results past that - I think at some point a chess
program should play its own game :P).

The book was created from classics.pgn, which is a collection of classic games I
found at the site of another author's program (I think Amy, but I'm not sure),
with side variations/comments/etc stripped out from the game, and also two games
which I couldn't seem to repair were removed.

Note that I didn't resolve collisions to any hash cells.  Rather than chaining
or doing linear probing and giving myself a headache (okay, okay, I admit, it's
not TOO hard and I'm just lazy :P), I just use the first occurance at that cell,
and discard anything found after that doesn't match the full hash key.

Also - rather than simply dumping the hash table I used to make the book with
into a binary file and using that, I simply looped through all actual positions
which occured (I culled out any moves occuring only once), and wrote each
position to the binary file - you can see the difference - only 380 K for 40 ply
rather than 24 MB!

Anyways, I'm sorry about the long post, but I thought that people would be
interested in it, and hopefully find it useful.

FYI - I will finally be releasing the current version of Faile (1.4) after I
finish up a readme, do some play testing to make sure nothing funky is going on,
and create some distribution packages.  The source of course will be released as
well, and it is much neater than the Faile 0.6 source.  Those of you who enjoyed
looking over the v 0.6 source (actually, quite a few people have used it for
starting their own program - I even recall somebody porting the whole thing to
VB to make some sort of chess program with a GUI .. :P) will find the new v 1.4
has many more features and is overall a much better program.

Incidentally - I don't have enough web space to put up the PGN file I used to
make the book, or a swack of PGN files for you to make books with either, so ..
what would be your preference for the default book?  I'm thinking 30 ply..

So - expect an official release of Faile 1.4 sometime within the next week (the
length of time this takes depends on how fast I can get some testing done.. :P)


Regards,
Adrien.




Book Creation Memory Allocation:
2097152 book hash entries * 12 bytes/entry = 24576 kb of RAM

Making a new book from input file classics.pgn.
(Max book ply of 10)
(Each "." represents 100 games processed)

.................................................. (5000)
.................................................. (10000)
.............................

Total games found: 12935

Book Hash Table Report
(Counter max is 4294967295)

Positions found: 11194
Number of Hash Collisions: 75
Positions occuring > 100 times: 156
50 < Positions occuring <= 100 times: 185
10 < Positions occuring <= 50 times: 899
1 < Positions occuring <= 10 times: 3853
Positions occuring 1 time: 6101

Book File Creation:
5093 book entries * 12 bytes/entry ~= 59 kb book size

==================================================

Book Creation Memory Allocation:
2097152 book hash entries * 12 bytes/entry = 24576 kb of RAM

Making a new book from input file classics.pgn.
(Max book ply of 20)
(Each "." represents 100 games processed)

.................................................. (5000)
.................................................. (10000)
.............................

Total games found: 12935

Book Hash Table Report
(Counter max is 4294967295)

Positions found: 73875
Number of Hash Collisions: 2431
Positions occuring > 100 times: 195
50 < Positions occuring <= 100 times: 252
10 < Positions occuring <= 50 times: 1891
1 < Positions occuring <= 10 times: 16960
Positions occuring 1 time: 54577

Book File Creation:
19298 book entries * 12 bytes/entry ~= 226 kb book size

==================================================

Book Creation Memory Allocation:
2097152 book hash entries * 12 bytes/entry = 24576 kb of RAM

Making a new book from input file classics.pgn.
(Max book ply of 30)
(Each "." represents 100 games processed)

.................................................. (5000)
.................................................. (10000)
.............................

Total games found: 12935

Book Hash Table Report
(Counter max is 4294967295)

Positions found: 160815
Number of Hash Collisions: 8967
Positions occuring > 100 times: 198
50 < Positions occuring <= 100 times: 254
10 < Positions occuring <= 50 times: 2025
1 < Positions occuring <= 10 times: 25426
Positions occuring 1 time: 132912

Book File Creation:
27903 book entries * 12 bytes/entry ~= 326 kb book size

==================================================

Book Creation Memory Allocation:
2097152 book hash entries * 12 bytes/entry = 24576 kb of RAM

Making a new book from input file classics.pgn.
(Max book ply of 40)
(Each "." represents 100 games processed)

.................................................. (5000)
.................................................. (10000)
.............................

Total games found: 12935

Book Hash Table Report
(Counter max is 4294967295)

Positions found: 229705
Number of Hash Collisions: 17506
Positions occuring > 100 times: 198
50 < Positions occuring <= 100 times: 253
10 < Positions occuring <= 50 times: 2012
1 < Positions occuring <= 10 times: 29559
Positions occuring 1 time: 197683

Book File Creation:
32022 book entries * 12 bytes/entry ~= 375 kb book size



This page took 0.1 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.