Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Tip: how to reduce hard drive churning with tablebases

Author: Anthony Cozzie

Date: 06:03:09 03/09/04

Go up one level in this thread


On March 09, 2004 at 07:22:43, Richard Pijl wrote:

>On March 08, 2004 at 23:49:58, William Penn wrote:
>
>>Pretty simple. Reduce hash size. That's the only thing I've found to have a
>>significant effect when tablebase access starts to churn the hard drive
>>constantly. Engine speed (kN/s) falls dramatically at that point, perhaps to 10%
>>or less of normal speed, and never recovers. However using smaller hash size
>>appears to fix this problem.
>>
>>For example my computer has 1G RAM installed. I can run Shredder 8 with 768MB
>>hash normally, although I often use 512MB which the op system prefers a bit
>>more. Now one would think that 512MB hash would be OK in any situation with 1G
>>RAM, but not so. It's too much hash when tablebase access starts to crank up
>>heavy in endgame situations. At that point, reducing hash size to 256MB usually
>>fixes this problem, restoring engine speed to a reasonable kN/s. I haven't yet
>>found it necessary to goto 128MB hash.
>>
>>[Windows XP Home, Athlon XP 2400+/2.0GHz, 1G RAM]
>>WP
>
>I think this is a correct observation.
>
>You have to be aware that EGTB's will be cached in a lot of different way. In
>the Baron I store probed positions in the Hashtable, there is the EGTB cache
>where segments of uncompressed EGTB entries are stored, and the Windows cache
>where compressed chunks of EGTB's are stored.
>
>If no caching would take place, I would expect knps to drop to less than 1/100
>of the speed during normal search (without EGTB's) as seek times of disks are
>still several ms. If 1 out of 10 innernodes would probe the EGTB's you would get
>no more than about 2-3 knps and that is very optimistic. It will probably be
>significantly less.
>
>Fortunately chess programs store, reuse or cache the done work in several ways,
>and with a different granularity.
>
>For one EGTB probe that results in a disk access Hashtables store 1 position,
>EGTB cache stores several positions (uncompressed) and the windows cache stores
>a large chunk of (compressed) data. The chance this data will be reused in a
>later probe is significant.
>
>The configuration I use for the Baron during ICC games or tournaments is that I
>have 400Mb of transposition hashtables and 64 MB of EGTB cache. Internal memory
>is 1 Gb. The Baron probes EGTB's in all inner nodes. In this configuration I saw
>the knps go _up_ in endgames, due to the saved cost in evaluation of leaf-nodes.
>I don't have SCSI disks. Just plain 7200rpm IDE disks (IBM Deskstar).

Wow, you use IBM Deathstars?  I've heard nothing but bad things about them
(crashes/lost data).

anthony

>The reason why I limit the hashtable to 400 Mb is a different one though. I
>still have Win2k prof as the OS. There the memoryaccess above 512Mb will not be
>cached, so chosing a hashtable size above 400Mb has a negative effect on regular
>search too. But for diskcache that is good enough ...
>
>Richard.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.