Author: Robert Hyatt
Date: 21:17:33 11/17/98
Go up one level in this thread
On November 17, 1998 at 22:34:30, Hristo wrote: >On November 17, 1998 at 20:48:03, Robert Hyatt wrote: > >>On November 17, 1998 at 20:30:07, Hristo wrote: >> >>>On November 17, 1998 at 17:19:30, Robert Hyatt wrote: >>> >>>>On November 17, 1998 at 15:34:15, Eugene Nalimov wrote: >>>> >>>>>Very rough estimation: >>>>> >>>>>Pawnless 6-man ending - 10**10 positions. >>>>>6-man ending with pawn(s) - 2.5*10**10 positions. >>>>> >>>>>8 bits per position is not enough, 9-10 bits necessary. >>>>>So, pawnless 6-man ending will use 20Gb, ending with pawn(s) >>>>>will use 50Gb. >>>>> >>>>>It's possible to handle huge files at PC using NT (there is >>>>>no such restriction on file size as in Linux), but to be >>>>>reasonable effective generator must use 1-2Gb of RAM (I'm >>>>>speaking not about my generator, but about generator that >>>>>is written especially with 6-man tables in mind). Also, you >>>>>must have necessary amount of disk space free. >>>> >>>> >>>>how do you address such a big file since (a) the PC has 32 bit words >>>>and (b) the ansi standard requires that the data type for fseek() be >>>>a "long" and not something obtuse? >>>> >>>>IE the alpha handles this nicely even under linux... but on a peecee... >>>> >>>> >>>> >>> >>> >>>Bob check your data ... and don't spread rumors, please! >>>You are a teacher after all you should know the truth, no?! >>>Eugene already said something abou this ... but here are the details. >>>This information has been available since the first release of WinNT. >> >> >>depends on what you mean by "should know the truth". I don't do windows, >>I don't do microsoft. I do ansi C. And as I mentioned, ansi and posix have >>a standard for the "fseek()" function call... as far as "setfilepointer" >>That may well work perfectly... doesn't do a lot for portability however, >>which is a huge issue with me of course... >> >>And it could have been available since the first release of win 2.0 for all I >>know. As I said, I am "unix-only" here and "ansi/posix standard only" as well. >> >>Bob >> >> > >This > >>>>how do you address such a big file since (a) the PC has 32 bit words > >point (a) means absolutely nothing ... however if somebody reads this, might be >left with the impression that PCs are simply incapable of addressing large files >... is this TRUE? >and this point (a) means a *lot*. Yes there are lots of non-standard and non-portable ways to "beat the 32 bit limit." The windows trick is one. Or do as I have done for years and create multiple files when 2 gigs (for those systems that use a signed long file offset) isn't enough... And you could *obviously* do whatever you want in assembly language, to take this to an extreme point... But it has nothing to do with what I would call "c"... as defined by the standards I go by (ansi or posix) > >>>>IE the alpha handles this nicely even under linux... but on a peecee... > >is what I consider missleading and "untrue". The PC is not the problem, right? >It is the implementation of fseek ... in ANSI C the PC is most of the problem... because it is a 32 bit architecture. And the portable facilities for accessing files are defined around the native word length... But if you look at my post, I wasn't saying "it can't be done." I specifically asked Eugene *how* it could be done... I was hoping for some portable trick... >I find it interesting that you "jump" on anybody that makes demeaning comments >about DB, even if those people were not fully aware of what was going on with >DB. Based on their "lack" of knowledge, and your extended knowledge you "seem" >to know the truth and want us to know it as well, or at least beleive you! This >is fine. However you don't seem to follow the same, based on absolute knowledge >and precise understanding, path of describing the problems when you are talking >about PC(peecee). It's interesting how easily you dissmis curtain inacurasies in >your postings. >I felt compeled to call you on this particular one because of all the *hoopla* >about DB. > Big difference. No one accused anyone here of "cheating" nor of being "dishonest". I doubt Eugene took my post as saying "he was wrong and this can't be done." I'd hope he saw the ? on the end and figured out I was simply stating a difficulty I see (and still see) and was asking how he would solve that... Nothing more... nothing less... no accusations... no insults... no exaggerations... etc... IE this has *zilch* to do with the deep blue threads... I'd be happy to discuss deep blue in a sane and calm manner. But some seem to have a strong dislike for the program/machine/team, and make statements that someone needs to challenge. I'm that someone as often as not... > >hristo > >> >> >>> >>>MSVC6.0-5.0-4.0 help .... Start >>> >>>SetFilePointer >>>The SetFilePointer function moves the file pointer of an open file. >>> >>>DWORD SetFilePointer( >>> HANDLE hFile, // handle of file >>> LONG lDistanceToMove, // number of bytes to move file pointer >>> PLONG lpDistanceToMoveHigh, >>> // pointer to high-order DWORD of >>> // distance to move >>> DWORD dwMoveMethod // how to move >>>); >>> >>>Parameters >>>hFile >>>Handle to the file whose file pointer is to be moved. The file handle must have >>>been created with GENERIC_READ or GENERIC_WRITE access to the file. >>>lDistanceToMove >>>Low-order 32 bits of a signed value that specifies the number of bytes to move >>>the file pointer. If lpDistanceToMoveHigh is not NULL, lpDistanceToMoveHigh and >>>lDistanceToMove form a single 64-bit signed value that specifies the distance to >>>move. If lpDistanceToMoveHigh is NULL, lDistanceToMove is a 32-bit signed value. >>>A positive value for lDistanceToMove moves the file pointer forward in the file, >>>and a negative value moves the file pointer backward. >>>lpDistanceToMoveHigh >>>Pointer to the high-order 32 bits of the signed 64-bit distance to move. If you >>>do not need the high-order 32 bits, this pointer may be NULL. When non-NULL, >>>this parameter also receives the high-order DWORD of the new value of the file >>>pointer. For more information, see the Remarks section later in this topic. >>>dwMoveMethod >>>Starting point for the file pointer move. This parameter can be one of the >>>following values. Value Meaning >>>FILE_BEGIN The starting point is zero or the beginning of the file. >>>FILE_CURRENT The starting point is the current value of the file pointer. >>>FILE_END The starting point is the current end-of-file position. >>> >>> >>>MSVC6.0-5.0-4.0 help ... End >>> >>> >>> >>> >>>hristo >>> >>> >>> >>> >>> >>> >>> >>>>> >>>>>The hardest part will be generation of pawnless endings - >>>>>endings with pawns can be partitioned to a lot of subclasses, >>>>>depending of the pawns locations. >>>>> >>>>>Of course you can >>>>>(1) Generate on a powerful supercomputer (BTW, task can be >>>>> parallelized very well), compress table, transfer compressed >>>>> table to PC, and decompress only small chunks when necessary. >>>>>(2) Generate not DTC or DTM, but win/draw/loss table - >>>>> then you can pack 5 positions per byte. >>>>>(3) Generate some subclasses - e.g. with rammed pawns, or >>>>> with 2 identical pieces, etc. >>>>> >>>>>Eugene >>>>> >>>>>On November 17, 1998 at 13:03:53, Robert Hyatt wrote: >>>>> >>>>>>On November 17, 1998 at 08:14:45, Mark Young wrote: >>>>>> >>>>>>>On November 17, 1998 at 08:05:33, Robert Hyatt wrote: >>>>>>> >>>>>>>>On November 17, 1998 at 06:35:08, Ralph E. Carter wrote: >>>>>>>> >>>>>>>>>Has anyone projected the size of these? >>>>>>>> >>>>>>>> >>>>>>>>easy to do... for edward's format, use 64* size of previous file. IE for >>>>>>>>openings with at least one pawn, two files each with a size of 32 gigabytes. >>>>>>>> >>>>>>>>Eugene's format is more compact, around 50%. Still *very* *big*... >>>>>>> >>>>>>>How big a factor increase is adding pawns to the table bases that are 6 ,7 or 8 >>>>>>>man. >>>>>> >>>>>>It really is not a factor. IE if you have a totally pawnless tablebase, then >>>>>>a tablebase with a single pawn is bigger, because some of the compression >>>>>>tricks can't be used or you will make a pawn move in impossible directions when >>>>>>you rotate reflect and mirror the board. But once you have a single pawn on >>>>>>the board, every additional piece is roughly 64 time bigger than the last >>>>>>one. Eugene actually beats this a bit, so that since some squares are occupied >>>>>>(say 5 in a 5 piece ending, going to a 6 piece ending only takes 59* as much >>>>>>space (if I calculated that right) and not 64* like the old Edwards indexing >>>>>>scheme... >>>>>> >>>>>>But it really is moot... No PC's allow files that large today, because of the >>>>>>32 bit nature of the processor... you can't go beyond 4 gigs on any machine I >>>>>>know of until you step up to the alpha/etc 64 bit architectures... Yes it is >>>>>>"possible" to go beyond 4 gigs on a PC, but it makes handling the file index >>>>>>very messy... and I don't know of a system that does it...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.