Author: Robert Hyatt
Date: 08:36:01 01/31/01
Go up one level in this thread
On January 30, 2001 at 22:03:51, J. Wesley Cleveland wrote: >On January 29, 2001 at 23:51:00, Robert Hyatt wrote: > >>On January 29, 2001 at 13:41:57, guy haworth wrote: >> >>>Rob >>> >>>My thoughts were that: >>> >>> a) 2-bit tables are 1/4 the size before compression >>> >>> b) compression is far more effective (as noted by the other reply) >>> especially as 'broken' positions need not be marked as such >>> they could 'falsely' be given the previous unbroken's value >>> this improves the compression a bit more >>> >>> c) you only go to the DTZ/DTR (DTC/DTM) tables on value-preserving moves >>> and maybe you get an 'optimal' in half the time or less >>> >>>You only have to have the '2-bit' and '8-bit' tables in play (in or near RAM) >>>for the relevant (sub-)endgame, taking into account P-positions, B-colour and >>>K-positions. >>> >>>Have I missed anything? >>> >>>G >> >> >>You may have overlooked how I probe. IE I often see TB hits with a total >>of 16 pieces (or more) on the board at the root position. How can I tell >>which TBs I will need until I probe them? And sucking in 2-bit tablebases >>is going to cost a bunch when we are talking about 2 gigabytes. The idea is >>fine for 4-piece files. But all the 5's are done and we are now working on >>6's. Forget it for 6's, completely. > >It seems like we're discussing two different subjects here. >1. Whether all of 2-bit tablebases will fit in ram (they won't). >2. Whether 2-bit tablebases will greatly speed up the search (which I find much >more interesting). > >I ran some quick stats on the .tbs files from pub/hyatt. > >There are 357 tablebases. > >29 are 100% one result (4 draws, 23 white to move and win). >45 more are 99.9% or greater one result. >101 more are 95% or greater one result. > >All of these would compress very well if they were 2-bit tablebases. This means >the cache would be much more efficient for them. I also suspect that the TBs >probed deep in the search tend to be ones with large material imbalances (it's >much easier to lose material than it is to exchange it evenly), which are >largely represented above. I understand what you are explaining. The problem is best seen if we take three separate cases: 3-4 piece files 5 piece files 6 piece files 3-4 piece files _never_ caused my search to slow down at all. And not just because of caching. They simply don't get probed so often because reducing the total material to 4 is rare. 5 pieces started to cause big problems. Steven Edwards made the KRPKR and promotion cases for me after I asked him to, and I ran into a serious problem when I started using them... if you probe them all the way to the q-search, the program slows down so badly you will actually lose enough depth to make mistakes and lose to a deeper-searching program without tablebases, because he will find ways to win your material that you can't see because you are going so slowly. 6 piece files make this hugely worse. Yes, you will cut the I/O by at least 1/4th due to the smaller size. Yes it might compress better. But even if the 6's compress 10 times better in win/lose/draw format, they might still total to > 1 terrabyte. This is one reason I am not using any 6's at all right now. Even though there are some that would be useful at times. As disk speeds continue to slowly improve, and memory continues to get larger, this will slowly start to work more effectively... This would not be hard to test in Crafty. It would not be difficult to probe every entry in a file, and convert it to win/lose/draw, and then create a set of files with just that data (I think Nimzo has this already for selected files). However, the question is "which files to do it to?" and I think the answer must be "all". Someone once asked me "which are the most important files to download?" I wasn't sure so I simply modified Eugene's code to keep up with which files were probed during a game. The answer? almost all of them. In games where any are probed. Yes, once you reach a KRP vs KRN you won't hit all, but well before you reach that point, you will touch everything... The idea seems pretty good for 3-4 piece files, and even for 5 piece files although the memory to hold them becomes prodigous. But 6's are hopeless as todays machines are no better at probing a 1gb file than they are at probing a 1 terrabyte file.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.