Author: Vincent Diepeveen
Date: 13:54:22 05/28/02
Go up one level in this thread
On May 28, 2002 at 03:30:53, Russell Reagan wrote: first of all the size problem is there indeed. whatever compression you use, it will always be big tables. However obviously generating them only has to be performed once, which means that most generators are not exactly speed based. a month a table at a fast big alpha is indeed the truth now i guess. Reality is that it can be done a lot of times faster with a faster generator. Take mine. Also nalimov/heinz format stores positions in 2 bytes a position or so at 6 men. I just store win/draw/loss/unknown when generating and win/draw/loss when finished generating. that's 16 bits versus less than 2 bits a position. Factor 8 in size difference directly (practical less than that as i use a less efficient scheme). However with compression using the superb compression it compresses much better, so the total size needed to store the EGTBs is much smaller. Doesn't take away that all 6 men represent a total size which is real *huge*. there are way more 6 men EGTBs than there are 5 men. The 5 men is a complete 'few hours' generation joke compared to what happens with the 6 men. If i have some spare time left this summer after world champs i will finish the en passant bug i have now and add the superb compression from andrew kadatch to the EGTBs and if i win the toto somewhere that i can afford to buy myself enough ram and a fast U160 SCSI harddisk or if someone has 2 GIGAbyte ram (windows or linux is no issue here. also other OSes no problem as it is ansi-c) and a big harddisk and willing to generate some 6 men, then please let him either shout loud or email me. 512mb ram or so which i have myself and many others is simply a big problem to generate 6 men with pawns on the board. It slows down the generation process at least 8 times. about 40 gigs of diskspace needed to generate a 6 men with a pawn like KRP KRP is a real interesting egtb. But before that also other EGTBs are needed. Thanks to andrew kadatch for compression. without it, it would have been hopeless. note that during generation the 6 men are a big pain in the ass with regards to number of files needed for it. Each file i create is not so big, so the total number of files during generation is real big and some which get dleted later need to be on disk for a while in order to prevent slow random lookups to other EGTBs. All these tricks and many not mentionned are needed in order to be able to generate 6 men. The file system has problems with them, how i go manage to compress many files to 1 compressed 6 man file is also a big mystery to me still and other issues are there such as that i timed how long it takes to initialize the EGTBs and i timed how long it took to do 1 pass for a 6 men with pawn, but the total time needed to generate all 6 men is not known. Nor do i know how well all 6 men compress to file sizes. Some real difficult 6 men might be there to compress. thinking of KRP KBN for example. Anyway, nothing limits us in generating 6 men except there are no volunteers with a fast harddisk and 2 GIGs of RAM to generate them. My plan was to wait till i could afford myself 2 gigs and a fast and real big SCSI harddisk. With current recession buying this is delayed till end of this year though :) Previous calls in this forum to generate 6 men no one answerred except several who wanted to try with 256MB ram and an IDE harddisk, which is like going to take years of course. Still thanks for the offer though. Also i remember Bob has 512MB ram and fast SCSI harddisks, yet i don't generate DTM. With DTM the 6 men will take years for sure. let's not discuss the 7 men. Those are *way* out of reach. At least for my generator. >Some of the 6-piece tablebases are over a gigabyte in size. Some of those files >take over a month to generate (assuming you even have hardware capable of >generating them in the first place), and there are still many more files to be >generated before the 6-piece tablebases are complete. > >So, I'm wondering how long we (we being Bob or whoever is doing this computing), >are going to continue generating these things. In a few years (or however long >it takes) when the 6-piece tablebases are complete, will we start on the 7-piece >tablebases? After those are complete many more years later, do we then start on >the 8-piece tablebases? > >It seems to me that eventually, since this problem will grow exponentially, that >it will cause a major problem somewhere down the line, because the size of hard >disks does not seem to be growing exponentially with respect to time. Eventually >we will get to the point where a single tablebase file will not fit onto any >hard disk. Eventually the exponential growth of the time it takes to produce the >files will grow beyond the computing power of the time. > >When that happens, will the tablebase generation continue on? Or will you look >at the problem and think, "well it took 30 years to generate the N-piece >tablebases, so it will take at least 10,000 years to generate the N+1-piece >tabelbases" and decide to stop? Or will you continue on in hopes that hardware >advances will help things move along more quickly? > >If you will eventually stop generation of the tablebases, when would you >estimate that will be? Not in years time, but after which level of tablebase >generation? After 8-piece tablebases are complete? Or 7-piece tablebases? Any >ideas? > >Russell
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.