Author: Mark Young
Date: 20:45:33 06/27/99
Go up one level in this thread
On June 27, 1999 at 23:21:14, Paul Richards wrote: >On June 27, 1999 at 15:29:07, Mark Young wrote: > >>On June 27, 1999 at 15:23:19, Robert Hyatt wrote: > >>>Crafty doesn't work well with ponder=off. An older version of Rebel would >>>really screw up as it only did its time calculation while in ponder mode.. Ed >>>and I ran into several such problems when we did that single "NPS game" a >>>year or two ago... >>> >>>In any case it definitely screws my time allocation up. And I'll bet other >>>programs play weaker than normal because none of us do a lot of testing with >>>no pondering, to make sure everything is working well. I don't test that way >>>because I never play games that way... >> >>That my be true with Crafty and Rebel, but not for the commercail programs that >>run inside chessbase, that I have been able to detect. > >I think that's the point, i.e. we have no idea what the commercial programs >do because it isn't made public. We also don't have the source code for >the Chessbase interface. Since we can imagine a number of X-factors >occurring on a single machine, memory allocation issues, cache issues, >and God knows what else, we have yet to create a reliable test >environment on a single machine. To have even a reasonable guarantee that >one-machine testing is valid, you would have to run a large number of games >on both one machine and two machines until you have a large enough sample >size to make a statement as to the quality of results. You are correct and is why I made such a statement. Any programmer can say my program needs to ponder, my program needs X amount of hash etc. This may be true to some extent, but the question is how much a change is this going to make, rating gain or loss. Then when you test and see no significant change that makes you say, there is somthing wrong here between the two data points.... I say I see nothing wrong with testing with one computer in the chessbase interface. And then that would >only be valid for the two chess programs tested, and only for the exact >versions tested. Since we can't even pin down the number of variables >that might have an influence when running two programs at the same time, >the gold standard is clearly separate machines.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.