Author: Koundinya Veluri
Date: 03:23:10 10/19/02
Go up one level in this thread
On October 19, 2002 at 03:33:56, martin fierz wrote: >On October 19, 2002 at 01:04:21, Koundinya Veluri wrote: > >>On October 19, 2002 at 00:39:19, martin fierz wrote: >> >>>On October 19, 2002 at 00:27:44, Koundinya Veluri wrote: >>> >>>>On October 18, 2002 at 18:43:15, Gian-Carlo Pascutto wrote: >>>> >>>>>On October 18, 2002 at 18:35:32, martin fierz wrote: >>>>> >>>>>>so your book doesn't fit into your memory? wow, that thing must be huge! or >>>>>>maybe it's just that i'm spoiled with my GB-machine :-) >>>>> >>>>>On my machine it does, but I don't want to make the assumption it >>>>>will on all machines, so the code uses fseek/fread/fwrite. >>>>> >>>>>I have 768M RAM. The book is 16M, the PGN is 153M. Windows is >>>>>*continously* busy with the disk and my program is getting <5% >>>>>CPU time. That's redicolous. >>>> >>>>If your pgn is too fragmented, that would explain the heavy disk access. Try >>>>defragmenting it. I've built books from some huge pgn files that were much >>>>larger than my ram and still had no problems. Even disabling caching completely >>>>shouldn't slow the program down that much. Probably your pgn file is fragmented >>>>on your windows partition and isn't fragmented on your linux partition. >>>> >>>>Koundinya >>> >>>if his book generator is doing what i think it should be doing (reading in one >>>PGN file after another, and writing the book to disk from time to time), then >>>the read operation is negligible compared to the write of 16MB (which is what he >>>says his book file size is). >>>there is no way that reading games from HD is going to take any time at all - >>>remember, GCP says he is only doing a few 10 games/s, and that kind of lousy >>>performance cannot be due to harddisk reading, however fragmented it is. >>> >>>aloha >>> martin >> >>Although I admit that I have no idea what his book builder is doing, I don't >>agree that reading 153 MB of formatted text data is negligible compared to >>writing 16 MB of unformatted binary data, especially since the program has to >>wait for each read operation to complete while it doesn't have to wait for a >>write operation to complete. I can't find another reason for that much disk >>access, but I assume it's not the writes that are clogging up because they can >>easily be cached while the reads cannot. You're probably right though that 10 >>games/s is too slow for only fragmentation to affect. >> >>Koundinya > >as far as i understand it, he is reading the 153MB once, but writing the 16MB >multiple (thousands of!) times. that's why i said the read was negligible. > >aloha > martin Oh I misunderstood that. Now I see what's going on. Koundinya
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.