Author: martin fierz
Date: 00:33:56 10/19/02
Go up one level in this thread
On October 19, 2002 at 01:04:21, Koundinya Veluri wrote: >On October 19, 2002 at 00:39:19, martin fierz wrote: > >>On October 19, 2002 at 00:27:44, Koundinya Veluri wrote: >> >>>On October 18, 2002 at 18:43:15, Gian-Carlo Pascutto wrote: >>> >>>>On October 18, 2002 at 18:35:32, martin fierz wrote: >>>> >>>>>so your book doesn't fit into your memory? wow, that thing must be huge! or >>>>>maybe it's just that i'm spoiled with my GB-machine :-) >>>> >>>>On my machine it does, but I don't want to make the assumption it >>>>will on all machines, so the code uses fseek/fread/fwrite. >>>> >>>>I have 768M RAM. The book is 16M, the PGN is 153M. Windows is >>>>*continously* busy with the disk and my program is getting <5% >>>>CPU time. That's redicolous. >>> >>>If your pgn is too fragmented, that would explain the heavy disk access. Try >>>defragmenting it. I've built books from some huge pgn files that were much >>>larger than my ram and still had no problems. Even disabling caching completely >>>shouldn't slow the program down that much. Probably your pgn file is fragmented >>>on your windows partition and isn't fragmented on your linux partition. >>> >>>Koundinya >> >>if his book generator is doing what i think it should be doing (reading in one >>PGN file after another, and writing the book to disk from time to time), then >>the read operation is negligible compared to the write of 16MB (which is what he >>says his book file size is). >>there is no way that reading games from HD is going to take any time at all - >>remember, GCP says he is only doing a few 10 games/s, and that kind of lousy >>performance cannot be due to harddisk reading, however fragmented it is. >> >>aloha >> martin > >Although I admit that I have no idea what his book builder is doing, I don't >agree that reading 153 MB of formatted text data is negligible compared to >writing 16 MB of unformatted binary data, especially since the program has to >wait for each read operation to complete while it doesn't have to wait for a >write operation to complete. I can't find another reason for that much disk >access, but I assume it's not the writes that are clogging up because they can >easily be cached while the reads cannot. You're probably right though that 10 >games/s is too slow for only fragmentation to affect. > >Koundinya as far as i understand it, he is reading the 153MB once, but writing the 16MB multiple (thousands of!) times. that's why i said the read was negligible. aloha martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.