Author: Miguel A. Ballicora
Date: 06:04:35 10/19/02
Go up one level in this thread
On October 19, 2002 at 06:23:10, Koundinya Veluri wrote: >On October 19, 2002 at 03:33:56, martin fierz wrote: > >>On October 19, 2002 at 01:04:21, Koundinya Veluri wrote: >> >>>On October 19, 2002 at 00:39:19, martin fierz wrote: >>> >>>>On October 19, 2002 at 00:27:44, Koundinya Veluri wrote: >>>> >>>>>On October 18, 2002 at 18:43:15, Gian-Carlo Pascutto wrote: >>>>> >>>>>>On October 18, 2002 at 18:35:32, martin fierz wrote: >>>>>> >>>>>>>so your book doesn't fit into your memory? wow, that thing must be huge! or >>>>>>>maybe it's just that i'm spoiled with my GB-machine :-) >>>>>> >>>>>>On my machine it does, but I don't want to make the assumption it >>>>>>will on all machines, so the code uses fseek/fread/fwrite. >>>>>> >>>>>>I have 768M RAM. The book is 16M, the PGN is 153M. Windows is >>>>>>*continously* busy with the disk and my program is getting <5% >>>>>>CPU time. That's redicolous. >>>>> >>>>>If your pgn is too fragmented, that would explain the heavy disk access. Try >>>>>defragmenting it. I've built books from some huge pgn files that were much >>>>>larger than my ram and still had no problems. Even disabling caching completely >>>>>shouldn't slow the program down that much. Probably your pgn file is fragmented >>>>>on your windows partition and isn't fragmented on your linux partition. >>>>> >>>>>Koundinya >>>> >>>>if his book generator is doing what i think it should be doing (reading in one >>>>PGN file after another, and writing the book to disk from time to time), then >>>>the read operation is negligible compared to the write of 16MB (which is what he >>>>says his book file size is). >>>>there is no way that reading games from HD is going to take any time at all - >>>>remember, GCP says he is only doing a few 10 games/s, and that kind of lousy >>>>performance cannot be due to harddisk reading, however fragmented it is. >>>> >>>>aloha >>>> martin >>> >>>Although I admit that I have no idea what his book builder is doing, I don't >>>agree that reading 153 MB of formatted text data is negligible compared to >>>writing 16 MB of unformatted binary data, especially since the program has to >>>wait for each read operation to complete while it doesn't have to wait for a >>>write operation to complete. I can't find another reason for that much disk >>>access, but I assume it's not the writes that are clogging up because they can >>>easily be cached while the reads cannot. You're probably right though that 10 >>>games/s is too slow for only fragmentation to affect. >>> >>>Koundinya >> >>as far as i understand it, he is reading the 153MB once, but writing the 16MB >>multiple (thousands of!) times. that's why i said the read was negligible. Now I do not understand. How is it possible to write 16MB thousands of times? Then it is trying to sort at the same time it is processing the entries? Miguel >> >>aloha >> martin > >Oh I misunderstood that. Now I see what's going on. > >Koundinya
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.