Author: Eugene Nalimov
Date: 08:46:32 04/02/04
Go up one level in this thread
Vincent, I did not gave you permission to use *my* code in your convertor, or I am missing something? Thanks, Eugene On April 02, 2004 at 11:15:56, Vincent Diepeveen wrote: >On April 01, 2004 at 22:30:27, Eugene Nalimov wrote: > >>On April 01, 2004 at 21:38:22, Marc Bourzutschky wrote: >> >>>On April 01, 2004 at 21:30:23, Eugene Nalimov wrote: >>> >>>>On April 01, 2004 at 21:15:05, Marc Bourzutschky wrote: >>>> >>>>>On April 01, 2004 at 20:36:32, Eugene Nalimov wrote: >>>>> >>>>>>On April 01, 2004 at 20:03:20, Marc Bourzutschky wrote: >>>>>> >>>>>>>On April 01, 2004 at 17:59:38, Eugene Nalimov wrote: >>>>>>> >>>>>>>>On April 01, 2004 at 15:16:34, Marc Bourzutschky wrote: >>>>>>>> >>>>>>>>>The Chessmaster format is indeed better >>>>>>>> >>>>>>>>What does it mean "better"? :-) >>>>>>>> >>>>>>>>It stores less information, thus compresses better. >>>>>>>> >>>>>>> >>>>>>>No, it stores the same information, just in two different files. Those files >>>>>>>together are still somewhat smaller than the Kadatch compressed ones... >>>>>> >>>>>>AFAIK it doesn't stores non-wins for side to move. Is it so? That alone should >>>>>>result in the better compression. >>>>>> >>>>> >>>>>It stores complete information, just distributed differently. For example, the >>>>>equivalent of the Nalimov krpkr in FEG is the set of krpkr and krkrp. Each >>>>>stores only wins for the side to move, but between the two you get the exact >>>>>same win/loss/draw information. krpkr and krkrp in FEG format togther occupy a >>>>>little under 127Mb, while the krpkr.nb?.emd files take 150Mb. The complete >>>>>5-man set in FEG is about 5.6Gb. >>>> >>>>Ok, let's assume that position in krpkr is loss for white to move. Where in FEG >>>>that information is stored? Not in krpkr, because it is not win for side to >>>>move. Not in krkrp because other side is to move. >>>> >>>>What I don't understand? >>>> >>> >>>Yes, the information is stored in krkrp "flipped" with black to move. In the >>>Nalimov format you don't need krkrp because krpkr stores both wins and losses, >>>while the FEG format does not need to store losses because they are in krkrp. >>>In fact, I used the 1-1 correspondence between Nalimov and FEG to trace an e.p. >>>bug in an earlier version of FEG :-) >> >>I see. So in FEG your have krpkr wtm, krpkr btm, krkrp wtm, and krkrp btm. Than >>yes, you can figure value without the search, at a cost of (probable) extra TB >>probe. Extra probe is unavoidable when the score is a draw. >> >>So your are paying that price, and slower access due to 4x larger block size, to > >I would agree with the term 'more efficient access'. > >Just admit that Johan has done a better job there and that i'm doing an even >better job there. > >>achieve ~30% smaller TBs. Reasonable tradeoff, but I would not call it "better" >>:-) > >For a factor 10 slowdown of your indexing code you save in your indexing scheme >about 10% positions. The biggest difference i saw so far was > >16.1G hardcoded entries for a 6 men yours, versus 18.1G entries for DIEP. > >By the way that 16.1G positions is compressed around 17GB in total (black & >white) and in diep format *uncompressed* 7.2 GB. > >You probably want to know what the sizes are compressed but i'm still busy >optimizing that. > >>BTW you can achieve better compression in .emd files by replacing all "broken" >>scores by the most common non-broken score in the TB. I always was curious how >>much it will save, but never made the experiment... > >I will save you from from crying and not show you the size of my EGTBs so far >converted. Let's put it this way, Kadatch is doing a brilliant work there. > >Note it's w/d/l so i should be 'only' factor 5 better anyway according to your >theories, isn't it? > >In general i'm amazed that someone whose job it is to make compilers which >produce efficient code, that he's doing such a bad job in coding himself. > >But well, just like me and Johan you do not get paid to create your egtb's. It's >all volunteer work with no one volunteering to generate them, even though >certain generators are pretty fast. I just lack the RAM to generate mine. > >>Thanks, >>Eugeen > >>>>Thanks, >>>>Eugene >>>> >>>>>>And as was pointed by Gian-Carlo Pascutto FEN format does not allow you to >>>>>>search in the tree -- i.e. it allows slower decompression. For example, you can >>>>>>achieve ~10% better compression with Kadatch algorithm just increasing block >>>>>>size from 8k to 16k. For larger block sizes you can achieve even better >>>>>>compression using slightly modified algorithm (modifications are useless with >>>>>>small block sizes). >>>>>> >>>>> >>>>>Yes, the block sizes may be an important factor. FEG uses 32k. How much of the >>>>>compression advantage would disappear with smaller block sizes only Johan de >>>>>Koning knows... >>>>> >>>>>-Marc >>>>> >>>>>>But I believe main reason for better compression is just less information in the >>>>>>files. >>>>>> >>>>>>Thanks, >>>>>>Eugene >>>>>> >>>>>>>-Marc >>>>>>> >>>>>>>>Thanks, >>>>>>>>Eugene >>>>>>>> >>>>>>>>>but no interface other than through >>>>>>>>>Chessmaster is available. Besides, the Nalimov format has become a quasi >>>>>>>>>industry standard. Since the contents of the two tablebases is the same, the >>>>>>>>>main advantage of the Chessmaster is faster generation with less RAM, and >>>>>>>>>somewhat smaller compressed file sizes. If there were a tool to translate >>>>>>>>>Chessmaster format to Nalimov format we would already have all the 6-man >>>>>>>>>tablebases by now. >>>>>>>>> >>>>>>>>>On April 01, 2004 at 14:18:16, Jason Kent wrote: >>>>>>>>> >>>>>>>>>>On April 01, 2004 at 14:07:22, Kurt Utzinger wrote: >>>>>>>>>> >>>>>>>>>>>On April 01, 2004 at 13:56:25, Jason Kent wrote: >>>>>>>>>>> >>>>>>>>>>>>I just read this in the FEG.txt that i got off the chessmaster website. >>>>>>>>>>>> >>>>>>>>>>>>XVI. WHY THE FEG FORMAT? ================================================ >>>>>>>>>>>> >>>>>>>>>>>>Perhaps, after reading all of this, you are wondering why Chessmaster >>>>>>>>>>>>9000 does not use either of the more commonly used EGDB formats, namely >>>>>>>>>>>>those created by Eugene Nalimov or Ken Thompson (both of which bear the >>>>>>>>>>>>names of their creators). There are many reasons for this: >>>>>>>>>>>> >>>>>>>>>>>>1. FEG data is about 20% smaller. Additionally, half-sets can be used if >>>>>>>>>>>> hard drive space is an issue, making a "full useable" file set that is >>>>>>>>>>>> almost 1/3 the size of the complete Nalimov file set. >>>>>>>>>>>>2. FEG generation is much faster and doesn't need a huge amount of free >>>>>>>>>>>> RAM to create a set of files. >>>>>>>>>>>>3. FEG can do any 6-man files on a 32-bit platform. >>>>>>>>>>>>4. The Thompson format is not a complete set (especially pawns on both >>>>>>>>>>>> sides are lacking). >>>>>>>>>>>>5. The Thompson format stores DTC (Distance to Conversion) values, >>>>>>>>>>>> meaning that it stores the number of moves to either mate OR to a >>>>>>>>>>>> capture/promotion, and will play whichever move has the smallest >>>>>>>>>>>> winning value. This can result in silly moves (a capture that leads to >>>>>>>>>>>> a mate in eight moves instead of a non-capture that leads to mate in >>>>>>>>>>>> three moves). >>>>>>>>>>>>6. Since Chessmaster 9000 is a mass market product, the majority of its >>>>>>>>>>>> users are not aware of these other formats and how to get them. Also >>>>>>>>>>>> for the ease of development it is easier not to be dependent on >>>>>>>>>>>> technical support for data that was created using tools that were not >>>>>>>>>>>> developed by Ubi Soft. >>>>>>>>>>> >>>>>>>>>>> Hi Jason >>>>>>>>>>> Maybe you have asked the wrong question: "Why does Chessmaster 9000 >>>>>>>>>>> not support the egtb format that all other engines do?". >>>>>>>>>>> Kurt >>>>>>>>>> >>>>>>>>>>The reason I ask is because the egtb format sounds like its a little better. I >>>>>>>>>>kinda wish cm9k used nalimov so it would be more compatible.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.