Author: Tony Werten
Date: 07:07:23 05/29/02
Go up one level in this thread
On May 29, 2002 at 09:18:43, Vincent Diepeveen wrote: >On May 29, 2002 at 01:07:21, martin fierz wrote: > >>On May 28, 2002 at 16:54:22, Vincent Diepeveen wrote: >> >>>On May 28, 2002 at 03:30:53, Russell Reagan wrote: >>> >>>first of all the size problem is there indeed. whatever compression >>>you use, it will always be big tables. >>> >>>However obviously generating them only has to be performed once, >>>which means that most generators are not exactly speed based. a month >>>a table at a fast big alpha is indeed the truth now i guess. >>> >>>Reality is that it can be done a lot of times faster with a faster generator. >>> >>>Take mine. >>> >>>Also nalimov/heinz format stores positions in 2 bytes a position or so >>>at 6 men. I just store win/draw/loss/unknown when generating and >>>win/draw/loss when finished generating. >>> >>>that's 16 bits versus less than 2 bits a position. Factor 8 in size >>>difference directly (practical less than that as i use a less >>>efficient scheme). However with compression using the superb compression >>>it compresses much better, so the total size needed to store the EGTBs >>>is much smaller. >>> >>>Doesn't take away that all 6 men represent a total size which is >>>real *huge*. there are way more 6 men EGTBs than there are 5 men. >>> >>>The 5 men is a complete 'few hours' generation joke compared to what >>>happens with the 6 men. If i have some spare time left this summer >>>after world champs i will finish the en passant bug i have now and >>>add the superb compression from andrew kadatch to the >>>EGTBs and if i win the toto somewhere that i can afford >>>to buy myself enough ram and a fast U160 SCSI harddisk >>> >>>or if someone has >>>2 GIGAbyte ram (windows or linux is no issue here. also other >>>OSes no problem as it is ansi-c) and a big harddisk and willing to >>>generate some 6 men, then please let him either shout loud or email >>>me. 512mb ram or so which i have myself and many others is simply >>>a big problem to generate 6 men with pawns on the board. It slows >>>down the generation process at least 8 times. >>> >>>about 40 gigs of diskspace needed to generate a 6 men with a pawn >>>like KRP KRP is a real interesting egtb. But before that also other >>>EGTBs are needed. >>> >>>Thanks to andrew kadatch for compression. without it, it would have >>>been hopeless. >> >>hi vincent, >> >>i recently computed the 8-piece (english) checkers endgame database, also just >>doing win-loss-draw. i used a 40GB HD / 1GB RAM system. i managed to compress it >>by about a factor of 9, approximately 20% better than what the chinook guys did >>- using a similar RLE scheme, which i tested a couple of times and then stopped >>working on after maybe a few evenings. i was wondering if RLE is in fact the >>best method to compress such databases, because back in 92 when schaeffer >>started this database thing, disk access was much faster compared to processors >>than today; so maybe today a scheme which compresses more but takes longer to >>decompress might be better. could you give some details of andrew's compression >>algorithm? i'd be very interested ;-) > >Though i have made my own huffman compressor and toyed with fractal >compression for half a month and also arithmatic compression, i will >be the first one to admit that i know near zero about the technical >capabilities of compression. > >Please explain some what is RLE? Run length encoding. What it basicly does is changing 444444 into 46 ( meaning 6 times a 4 ) Works great for drawish tablebases, specially if you change unknown and broken into draw. Tony > >You can see how andrew kadatch compression works 'easily' perhaps by >looking to nalimov code. nalimov code uses also kadatch compression. > >>aloha >> martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.