Author: Robert Hyatt
Date: 18:18:51 05/05/04
Go up one level in this thread
On May 05, 2004 at 20:37:23, Vincent Diepeveen wrote: >On May 05, 2004 at 19:36:23, Robert Hyatt wrote: > >>On May 05, 2004 at 19:22:40, Vincent Diepeveen wrote: >> >>>On May 05, 2004 at 16:44:59, Robert Hyatt wrote: >>> >>>>On May 05, 2004 at 16:28:35, Vincent Diepeveen wrote: >>>> >>>>>On May 05, 2004 at 11:56:21, Robert Hyatt wrote: >>>>> >>>>>>On May 05, 2004 at 09:29:19, Brian Kostick wrote: >>>>>> >>>>>>>On May 05, 2004 at 09:14:50, Mike Hood wrote: >>>>>>> >>>>>>>>On May 05, 2004 at 08:12:44, Vincent Diepeveen wrote: >>>>>>>> >>>>>>>>>On May 05, 2004 at 07:47:57, Mike Hood wrote: >>>>>>>>> >>>>>>>>>>I just let Filemon run while loading Fritz 8 to see why it takes so long. I was >>>>>>>>>>shocked to see that during the initialisation Fritz tries to open every possible >>>>>>>>>>tablebase. For instance... >>>>>>>>>> >>>>>>>>>>Open kpk.nbw.emd -- good, it's there >>>>>>>>>>Open kpknbw.emd -- file not found >>>>>>>>>>Open kpk_nbw.emd -- file not found >>>>>>>>>>Open kpk_nbw_emd -- file not found (I never knew this format was valid) >>>>>>>>>>Open kpk.nbw -- file not found >>>>>>>>>> >>>>>>>>>>And the same five accesses for the nbb file. >>>>>>>>>> >>>>>>>>>>Why carry on with the other three after finding the first tablebase? But it gets >>>>>>>>>>even wilder when it comes to the 6-piece tablebases. All 365 possible tablebase >>>>>>>>>>pairs in all possible formats are accessed, even though I don't have any on my >>>>>>>>>>disk. Thousands of "file not found" results. Just one example, to show how >>>>>>>>>>ludicrous it is: >>>>>>>>>> >>>>>>>>>>First Fritz tries to open krbnkp.nbw.emd, krbnkpnbw.emd, krbnkp_nbw.emd and >>>>>>>>>>krbnkp.nbw.emd. Almost the same as before, except Fritz is assuming 6-piece >>>>>>>>>>tablebases are compressed. But then Fritz tries to open krbnkp.0.nbw.emd, >>>>>>>>>>krbnkp.0_nbw.emd, krbnkp.0nbw.emd and krbnkp.0_nbw_emd. Then krbnkp.1.nbw.emd, >>>>>>>>>>etc... and krbnkp.2.nbw.emd... and all the way through to krbnkp.g.nbw.emd. That >>>>>>>>>>means 136 disk accesses for a tablebase that I don't have! And that's only one >>>>>>>>>>tablebase out of 365. >>>>>>>>>> >>>>>>>>>>Wouldn't it be much easier just to scan the tablebase directory and only open >>>>>>>>>>the files that actually exist? >>>>>>>>> >>>>>>>>>Both nalimov and i do this in a similar way. >>>>>>>>> >>>>>>>>>If you are willing to write code for this that works faster and works both for >>>>>>>>>windows and *nix, then i will be real happy to use it. >>>>>>>>> >>>>>>>>>Best Regards, >>>>>>>>>Vincent >>>>>>>> >>>>>>>>Thanks for the info, Vincent. I assumed the initialization code had been written >>>>>>>>by Chessbase, not by Eugene. >>>>>>>> >>>>>>>>My math was a bit off in my original post, but after looking at Filemon's log I >>>>>>>>can give the exact figure: Fritz attempts to access 33647 non-existent tablebase >>>>>>>>files. And please... you can't tell me that if the file krbnkp.0.nbw.emd doesn't >>>>>>>>exist it still makes sense to look for krbnkp.1.nbw.emd, krbnkp.2.nbw.emd, etc >>>>>>>>all the way to krbnkp.g.nbw.emd. That's a waste of processor time on any >>>>>>>>operating system. >>>>>>> >>>>>>> >>>>>>>Could you report the time from the first tb until the last? I know it will >>>>>>>change from system to system but curious just to know how long it takes there. >>>>>>>BK >>>>>> >>>>>> >>>>>>About 10 seconds on my box with about 1/2 terabyte of egtbs in use, maybe a tad >>>>>>less than 1/2 terabyte.. >>>>> >>>>>About 3 hours for a 512 processor supercomputer though with 512GB ram and 1 TB >>>>>i/o. >>>> >>>>So? It would take about 1 second on a Cray T932. >>>> >>>>But how would you know since you didn't have any? More made-up numbers??? >>>> >>>>I can open all the 6-piece files over NFS in a couple of minutes on my laptop... >>> >>>Oh Cray would crash when initalizing at 512 processors the egtb's. It doesn't >>>have that many file handles in its OS as you must locally initialize >>>everything... >> >>You don't need that many file handles on the Cray I mentioned. It is fully SMP >>shared memory. N files == N file handles, even though not all are kept open all >>the time... >> >>What you mean by locally on a T90 I have absolutely no idea... > >You talk about 16 shit processors or so from a 1Ghz. So? I was pointing out your stupid 3 hour time to open all the files is just beyond ridiculous. I've run on SGI hardware. We have a challenge at our supercomputer center. It's I/O is _nowhere_ near as bad as you imply. It could open all those files far faster than my dual xeon, for example... > >I talk about a more modern 512 processor Cray made out of alpha 21264's :) Not shared memory. Who cares? > >You never grow up. Talking about machines from the past which do not even boot >anymore :) You never grow up. Talking about numbers that only exist in your head and nowhere in reality. :) > >Next cray is in fact an opteron cluster created out of each node = dual opteron. > Not "the next Cray". Just "a cray". >They wanted first to use the HT to connect to other nodes, but didn't succeed >yet in that, so the in between model is using pci-x cards. > >When they get HT to work for the interconnects now *that* will kick some butt :) > >But even then diep would run *great*. The highend pci-x cards are 3.5 us latency >from myrinet (one way pingpong), so a full latency to get a hashtable position >is like 7 us only... Depends on whether they choose to do shared memory or not. T3d/T3e is definitely _not_ shared memory. MPI/PVM only. > >Now try to run full SMP on that :) > Why?? >> >>> >>>This 1024 processor R14000 replaced in fact a huge old Cray that we see you post >>>so much about...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.