Author: Angrim
Date: 15:26:58 02/11/02
Go up one level in this thread
On February 11, 2002 at 10:31:08, Robert Hyatt wrote: >On February 11, 2002 at 03:35:14, Angrim wrote: > >>On February 10, 2002 at 23:37:54, Robert Hyatt wrote: >> >>>On February 10, 2002 at 18:55:59, Angrim wrote: >>> >>>>On February 09, 2002 at 22:35:54, Robert Hyatt wrote: >><snip> >>>>>Several issues: >>>>> >>>>>1. 6 piece files are going to require some large mate-in-N scores. IE they >>>>>will need 16 bits at least, which doubles the size compared to a current 5 >>>>>piece file where all scores are 8 bits max. >>>> >>>>This doubles the size of the file before compression, but has little >>>>effect on the size of the compressed file. At least as long as the >>>>majority of the positions in the file are relatively short mates or >>>>draws. Only the positions which are long mates will take extra >>>>space after compression. Compression could be further improved by >>>>switching to building DTC tables. >>>> >>> >>>Think about that a moment, and you will see it is a flawed statement. IF >>>the file is 2x larger _before_ compression it will most likely be 2x larger >>>_after_ compression. Otherwise you are saying that larger files compress >>>more than smaller files which is a simply flaw in reasoning. >> >>I am saying that these specific 2x larger files will compress far >>more than the equivalent smaller files. Storeing a "5" in 2 bytes >>does not change how many bits long its corresponding huffman code is. >>It seems that you either forgot to "think about that a moment" before >>you advised me to do so, or you have never studied even the basics of >>file compression. >> >>As a simple test, try building one of the 4 or 5pc files using 16 bits >>per position, and compress that. Compare the resulting file size to >>the size of the same file built with 8bits per position. >>One experiment can save a lot of argument. >> >> >>>>>2. another piece multiplies the size by roughly 60. >>>> >>>>yep. >>>>>3. Unknown compression ability. We already have some 6 piece files that >>>>>are 3+ gigabytes. These are pawnless which have max symmetry. Pawns will >>>>>increase that significantly. >>>> >>>>I assume that is 3gig before compression. I would expect that >>>>6pc tables with one pawn(ie. KRPvKRN as a possible worst case) to >>>>take around 3GB compressed. >>> >>> >>>No.. That is 3 gigs _after_ compression. >> >>My mistake, I was thinking about the size of only the wtm half >>of the table. My estimate for KRPvKRN should have been 6GB then. >>And having seen Eugene's post with the actual size for KRBvKRN, >>I see that he is getting a little less compression than I had >>estimated, I would have figured 2.8GB for both sides of that >>one. So boost my estimate for KRPvKRN up to 7GB compressed for >>both sides. >> >>Angrim > > >I am talking about a _single_ file. _one_. Either wtm or btm. And we have >at least one that is just over 3 gigs already. Again, _one_ file. Not one >pair of files. That suprises me. Since the largest pawnless 6pc table is roughly 6.5GB uncompressed per side, this means that you are only getting a 2.2:1 compression ratio for this file. The worst that I have seen for the 5pc egtb is roughly 4:1 ratio. So I guess you are right about the "Unknown compression ability" for files with more pieces. >As far as the compression goes, what says that _most_ of the data will have >leading zeroes? I expect that most of the positions will be either draw, or mate in less than 127. Based on the 6pc tables which have been done so far(that I have seen the stats for at least), this seems reasonable. I agree that tables with pawns are likely to have higher average mate depths, but even then I expect the majority of the positions to be draws or "simple" mates in less than 127. For the remaining "complex" positions, I expect the leading digit will in general be a 1(mate in 128-255) or 2(mate in 256-383) If this is easy for you to check, I would be interested to know if any of the 6pc tables completed so far have more than 5% of the positions being mate in more than 100 turns. Also, for that file that is 3GB, what percent of the mates in it were deeper than 50,70, and 90 ply. Angrim ps. I just tried to go to ftp.cis.uab.edu to see if there are some statistics files there to look at, but the ftp server seems to be down right now.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.