Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Tip: how to reduce hard drive churning with tablebases

Author: Dieter Buerssner

Date: 11:48:56 03/09/04

Go up one level in this thread


On March 09, 2004 at 13:14:35, Mike Hood wrote:

>On March 08, 2004 at 23:49:58, William Penn wrote:
>
>>Pretty simple. Reduce hash size. That's the only thing I've found to have a
>>significant effect when tablebase access starts to churn the hard drive
>>constantly. Engine speed (kN/s) falls dramatically at that point, perhaps to 10%
>>or less of normal speed, and never recovers. However using smaller hash size
>>appears to fix this problem.
>
><-- snip -->
>
>I understand what you're saying. Don't fill up all the available memory for hash
>tables. Leave enough memory for Windows to use as file cache, so that the
>tablebase files are in memory. Right?
>
>If the code is wriiten efficiently, this method of yours shouldn't bring an
>improvement, and it might even make things worse. Maybe someone can comment on
>this who knows more about the tablebase access code. Once a tablebase access has
>been carried out, the result of the tablebase lookup should be stored in the
>hash tables. The next time the same position is needed, the chess program
>doesn't need another tablebase lookup, because the result is already in the hash
>tables. The space needed for a stored position in the hash tables is less than
>the blocks of data needed for a chunk of the tablebase file in the system cache,
>so the hash table storage should be more efficient than the system cache.

Yes. However it is not that easy. I also made the observation, that leaving
little RAM to the OS in TB-access positions can make things much slower, and
that it is often an advantage to reduce hash size. With the same argument, you
could say, keep Nalimov cache to zero and rather use the space as hash table. To
me it looks more or less obvious, that this would not be the case, and that some
(smaller) amount of RAM given to the Nalimov cache should speed things up in
practice. Similar, if you leave the OS some RAM for caching.

The Nalimov cache uses blocks of 8 kB (uncompressed). These fit typically on 2
kB on disk, so the same size OS cache shoudl be able to hold more data (but it
must be decompressed at each access - which on modern computers doesn't include
a too big overhead).

When you really have lots of RAM (say >= 1 GB) naturally the effects will be
smaller in practice. 700 or 800 MB of hash table should not make a big
difference per se, and there will be not much pressure to use all the bytes
available. On my older computer, with 64 MB RAM it could make a really
significant difference, to leave some MBs (say 20) for the OS for caching. It
was faster, than using the 20 MB as Nalimov cache alone.

Regards,
Dieter



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.