Author: Koundinya Veluri
Date: 22:04:21 10/18/02
Go up one level in this thread
On October 19, 2002 at 00:39:19, martin fierz wrote: >On October 19, 2002 at 00:27:44, Koundinya Veluri wrote: > >>On October 18, 2002 at 18:43:15, Gian-Carlo Pascutto wrote: >> >>>On October 18, 2002 at 18:35:32, martin fierz wrote: >>> >>>>so your book doesn't fit into your memory? wow, that thing must be huge! or >>>>maybe it's just that i'm spoiled with my GB-machine :-) >>> >>>On my machine it does, but I don't want to make the assumption it >>>will on all machines, so the code uses fseek/fread/fwrite. >>> >>>I have 768M RAM. The book is 16M, the PGN is 153M. Windows is >>>*continously* busy with the disk and my program is getting <5% >>>CPU time. That's redicolous. >> >>If your pgn is too fragmented, that would explain the heavy disk access. Try >>defragmenting it. I've built books from some huge pgn files that were much >>larger than my ram and still had no problems. Even disabling caching completely >>shouldn't slow the program down that much. Probably your pgn file is fragmented >>on your windows partition and isn't fragmented on your linux partition. >> >>Koundinya > >if his book generator is doing what i think it should be doing (reading in one >PGN file after another, and writing the book to disk from time to time), then >the read operation is negligible compared to the write of 16MB (which is what he >says his book file size is). >there is no way that reading games from HD is going to take any time at all - >remember, GCP says he is only doing a few 10 games/s, and that kind of lousy >performance cannot be due to harddisk reading, however fragmented it is. > >aloha > martin Although I admit that I have no idea what his book builder is doing, I don't agree that reading 153 MB of formatted text data is negligible compared to writing 16 MB of unformatted binary data, especially since the program has to wait for each read operation to complete while it doesn't have to wait for a write operation to complete. I can't find another reason for that much disk access, but I assume it's not the writes that are clogging up because they can easily be cached while the reads cannot. You're probably right though that 10 games/s is too slow for only fragmentation to affect. Koundinya
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.