Author: Robert Hyatt
Date: 14:31:13 10/06/99
Go up one level in this thread
On October 06, 1999 at 16:38:48, blass uri wrote: >On October 06, 1999 at 15:30:48, Robert Hyatt wrote: > > ><snipped> >>4. A program (ie chessmaster) might poll for input, consuming 1/2 of the cpu >>even though it is not 'thinking'. > >This is easy to check and if the testers are intelligent they will not let it. > >I remember from previous posts some monthes ago that it is not a problem in >Didzis Cirulis's games because he gives the opponents of chessmaster more time >becuase of this problem. > >It is not a problem in Fritz engine-engine games because the programs are not >slowed down in engine-engine games. >> >>There are _too_ many things a program _might_ do. I'll bet not one person >>gave any thought to a "learning cycle" in crafty, yet it does it in every game. >>And it steals 2-4 seconds of time from the opponent. In short time controls, >>that might be important. >> >>If I _know_ people are testing like this, I'll bet I can raise Crafty's rating >>by 100 points minimum. > >If you care that the opponent will be slower people are going to see it and >give the opponent more time. > >Testers can check if one program is weaker relative to regular games when the >opponent use 0 seconds per move. > > I won't say how, but it shouldn't take too much >>imagination to figure it out. :) And with that said, why bother testing in a >>way that is obviously potentially unreliable. > >The reason is very simple. >Many testers have not 2 machines with the same speed so how do you want them to >test? > >Uri The answer is simple. If you only have two test tubes in your lab, you can't do many chemical experiements because you contaminate one with the other. So if you only have _one_ test tube, you probably aren't going to do any useful testing so far as typical chemistry goes, and you just don't do it. You didn't mention the other things I wrote about in another post here. What about those fast games that are getting played, where after making a move crafty will spend a couple of seconds updating the book learning stuff. How will a program with a target time of (say) 2 seconds act if Crafty suddenly computes for 2-4 seconds _after_ it makes a move, which steals time from the other program. How to stop that? how to _detect_ that? This form of testing is simply flawed...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.