Author: martin fierz
Date: 10:44:17 10/19/02
Go up one level in this thread
On October 19, 2002 at 09:04:35, Miguel A. Ballicora wrote: >On October 19, 2002 at 06:23:10, Koundinya Veluri wrote: > >>On October 19, 2002 at 03:33:56, martin fierz wrote: >> >>>On October 19, 2002 at 01:04:21, Koundinya Veluri wrote: >>> >>>>On October 19, 2002 at 00:39:19, martin fierz wrote: >>>> >>>>>On October 19, 2002 at 00:27:44, Koundinya Veluri wrote: >>>>> >>>>>>On October 18, 2002 at 18:43:15, Gian-Carlo Pascutto wrote: >>>>>> >>>>>>>On October 18, 2002 at 18:35:32, martin fierz wrote: >>>>>>> >>>>>>>>so your book doesn't fit into your memory? wow, that thing must be huge! or >>>>>>>>maybe it's just that i'm spoiled with my GB-machine :-) >>>>>>> >>>>>>>On my machine it does, but I don't want to make the assumption it >>>>>>>will on all machines, so the code uses fseek/fread/fwrite. >>>>>>> >>>>>>>I have 768M RAM. The book is 16M, the PGN is 153M. Windows is >>>>>>>*continously* busy with the disk and my program is getting <5% >>>>>>>CPU time. That's redicolous. >>>>>> >>>>>>If your pgn is too fragmented, that would explain the heavy disk access. Try >>>>>>defragmenting it. I've built books from some huge pgn files that were much >>>>>>larger than my ram and still had no problems. Even disabling caching completely >>>>>>shouldn't slow the program down that much. Probably your pgn file is fragmented >>>>>>on your windows partition and isn't fragmented on your linux partition. >>>>>> >>>>>>Koundinya >>>>> >>>>>if his book generator is doing what i think it should be doing (reading in one >>>>>PGN file after another, and writing the book to disk from time to time), then >>>>>the read operation is negligible compared to the write of 16MB (which is what he >>>>>says his book file size is). >>>>>there is no way that reading games from HD is going to take any time at all - >>>>>remember, GCP says he is only doing a few 10 games/s, and that kind of lousy >>>>>performance cannot be due to harddisk reading, however fragmented it is. >>>>> >>>>>aloha >>>>> martin >>>> >>>>Although I admit that I have no idea what his book builder is doing, I don't >>>>agree that reading 153 MB of formatted text data is negligible compared to >>>>writing 16 MB of unformatted binary data, especially since the program has to >>>>wait for each read operation to complete while it doesn't have to wait for a >>>>write operation to complete. I can't find another reason for that much disk >>>>access, but I assume it's not the writes that are clogging up because they can >>>>easily be cached while the reads cannot. You're probably right though that 10 >>>>games/s is too slow for only fragmentation to affect. >>>> >>>>Koundinya >>> >>>as far as i understand it, he is reading the 153MB once, but writing the 16MB >>>multiple (thousands of!) times. that's why i said the read was negligible. > >Now I do not understand. How is it possible to write 16MB thousands of times? >Then it is trying to sort at the same time it is processing the entries? > >Miguel for some reason GCP doesn't want to assume the user has 16MB of memory, so he generates his book on disk. as far as i understand... aloha martin > > > >>> >>>aloha >>> martin >> >>Oh I misunderstood that. Now I see what's going on. >> >>Koundinya
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.