Author: Ed Schröder
Date: 03:07:49 10/07/99
Go up one level in this thread
>Posted by Robert Hyatt on October 06, 1999 at 15:30:48: > >I'm going to make a list of all the reasons why two programs, one computer, >is a bad thing to do: > >1. a program might not be well-adjusted in how it uses its time when it is >not allowed to "ponder". Crafty is an example. > >2. a program might not be well-behaved and do some unexpected computation >after it sends the move to the referee program. IE in crafty, I send the move, >then I do the learning stuff after 10 non-book moves have expired. This 'learning >cycle' can take 2-3-4 seconds with a really large book and a long opening line >in the book. Imagine what that does to a game/1minute time control that many >are using in winboard/xboard? > >3. a program (ie crafty) might do other things after it annouces its move, >such as malloc()'ing a large buffer for (say) learning or whatever. What does a >large malloc() do to the other program? swap it out? Never thought of this. This is a real killer, I agree. If done on purpose you win every game :-) >4. A program (ie chessmaster) might poll for input, consuming 1/2 of the cpu >even though it is not 'thinking'. Another true point. Every program needs to poll for input. Asking the keyboard / mouse for input are expensive (slow) operations. In Rebel I have a counter that makes sure to look for input after 500 evaluations. If I decrease the value to say 50 or 10 the NPS of Rebel drops tremendously (forgot about the exact slow-down). Thus the opponent program will ALWAYS slow-down yours and you can only guess how much that is. >There are _too_ many things a program _might_ do. I'll bet not one person >gave any thought to a "learning cycle" in crafty, yet it does it in every >game. And it steals 2-4 seconds of time from the opponent. In short time >controls, that might be important. > >If I _know_ people are testing like this, I'll bet I can raise Crafty's rating >by 100 points minimum. I won't say how, but it shouldn't take too much >imagination to figure it out. :) And with that said, why bother testing in a >way that is obviously potentially unreliable. For fun, sure. But reporting >the results as "A beats B" is not very scientific... A might not actually >be able to beat B, he just might have a smarter programmer that takes >advantage of a flawed testing methodology... > >:) > >Bob Strong points Bob. Still people are in love with the system as you only need one PC and have a lot of fun. If only its results are judged in the way it should. And in no way you can compare eng-eng matches on 1 PC with eng-eng matches on 2 PC's. Ed PS, match score sofar PB_ON vs PB_OFF 17.5 - 8.5
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.