Author: Robert Hyatt
Date: 13:25:58 02/11/02
Go up one level in this thread
On February 11, 2002 at 13:23:46, Heiner Marxen wrote: >On February 11, 2002 at 10:37:45, Robert Hyatt wrote: > >>On February 11, 2002 at 07:08:41, Heiner Marxen wrote: >> >>>On February 10, 2002 at 23:40:06, Robert Hyatt wrote: >>> >>>>On February 09, 2002 at 22:49:22, Russell Reagan wrote: >>>> >>>>>On February 09, 2002 at 22:35:54, Robert Hyatt wrote: >>>>> >>>>>>Several issues: >>>>>> >>>>>>1. 6 piece files are going to require some large mate-in-N scores. IE they >>>>>>will need 16 bits at least, which doubles the size compared to a current 5 >>>>>>piece file where all scores are 8 bits max. >>>>> >>>>>Why would you need to use a full 16-bits? And why would you add on "at least" to >>>>>that? How many mates are going to take more moves than 65,536 moves without >>>>>reaching a 50-move rule point or 3-fold repitition? The answer is none. Maybe >>>>>you could explain why my reasoning doesn't work. >>>>> >>>> >>>> >>>>Simply because the _current_ way of building them uses distance to mate. >>>>And distance to mate is distance to mate, period... and it will definitely >>>>push beyond 8 bits, and using 9 bits is _not_ convenient for reasonable speed >>>>and accessing. >>> >>>I feel different. Accessing non-byte aligned entities packed into memory >>>just needs a multiplication, a division and some shifting and masking. >>>That is not much overhead compared to the complex index computations, >>>with lots of table lookups, the bookkeeping of the caching and multiple >>>function calls done for just one EGTB probe. >> >>Based on that, why doesn't everyone use _any_ size for their hash tables, >>rather than using a power of 2 so that AND can extract the address part? > >Hello Bob, > >I'm not sure. In my program "chest" I _do_ use arbitrary sizes. >The additional %= size instead of &= (size-1) is not significant (for me). >The positive side is that I can use any (odd) amount of RAM available. > >>I'm not saying that using 9 or 10 or 12 bits is impossible. > >Ok, fair enough. > >> It is just messy >>as it produced "clumps" of data that have to go together. > >I'm not sure I understand. Some clustering can be done, but need not be. The compression scheme used at present works for 8 bit values. When they start to span then it won't be as effective, and since things are read/written on byte boundaries, the size of blocks will have to be a multiple of some odd number which is not a constraint right now.. Not impossible by any stretch, but definite coding for someone. And I suspectr someone == Eugene. :) > >> And that makes the >>compression/decompression messier. > >That is definitely true. Until we find a compression/decompression sceme >which is as effective for odd sized entities as the current one is for byte >oriented data. > >May be that the current compression/decompression algorithm can be modified >accordingly. Last time I looked into it I understood only part of it, >so I can only guess. > >> And that makes the probes more expensive. > >I think the probing would be slower, but not significantly so. >I would even expect the better usage of available memory (thereby caching >more effective data per MB) would outweigh that effect. >In total it could even be faster due to some saved disk accesses. That is possible of course. It works that way right now (compressed is better due to reduced I/O bandwidth). > >>Once they are built with 16 bits then _anyone_ can re-build them to 10 bits if >>that is enough and it won't take very long to just copy from one format to >>another. > >Without compression, yes. >With the current compression sceme 10-bit data would compress much worse. > >> It will just make the indexing calculations messier than they already >>are, and they are already very messy with the symmetry and other things going >>on... > >No, not much. I would not change the indexing at all. I would just change >the meaning of the index: instead of directly accessing a byte with that offset >(or two bytes at 2*index and 2*index+1) a not so complicated div/mod operation >would translate the (logical) index into a (physical) byte offset and a >bit offset. >The current index calculation is already significantly more expensive, IMHO. > >Currently all this might not be worth the hassle (changing software, having >yet another format) but in principle it could be an option for the future. > >Cheers, >Heiner > > > >>>OTOH, the memory savings can be significant. Also, when less than 8 bits >>>would be sufficient. >>> >>>I think, one main reason to stay byte aligned is that the compression >>>algorithm works on bytes, and would not achieve good compression, >>>when odd bit lengths are packed together, crossing byte boundaries all along. >>> >>>Unfortunately, I do not know of a better compression algorithm, which works >>>well with entities of arbitrary bit sizes. Otherwise I would already have >>>proposed it. >>> >>>Cheers, >>>Heiner
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.