Author: blass uri
Date: 09:37:45 10/08/99
Go up one level in this thread
On October 08, 1999 at 08:26:35, Robert Hyatt wrote: >On October 08, 1999 at 03:26:39, Peter Kappler wrote: > >>On October 07, 1999 at 14:12:53, Robert Hyatt wrote: >> >>>On October 07, 1999 at 06:07:49, Ed Schröder wrote: >>> >>>>>Posted by Robert Hyatt on October 06, 1999 at 15:30:48: >>>>> >>>>>I'm going to make a list of all the reasons why two programs, one computer, >>>>>is a bad thing to do: >>>>> >>>>>1. a program might not be well-adjusted in how it uses its time when it is >>>>>not allowed to "ponder". Crafty is an example. >>>>> >>>>>2. a program might not be well-behaved and do some unexpected computation >>>>>after it sends the move to the referee program. IE in crafty, I send the move, >>>>>then I do the learning stuff after 10 non-book moves have expired. This 'learning >>>>>cycle' can take 2-3-4 seconds with a really large book and a long opening line >>>>>in the book. Imagine what that does to a game/1minute time control that many >>>>>are using in winboard/xboard? >>>>> >>>>>3. a program (ie crafty) might do other things after it annouces its move, >>>>>such as malloc()'ing a large buffer for (say) learning or whatever. What does a >>>>>large malloc() do to the other program? swap it out? >>>> >>>>Never thought of this. This is a real killer, I agree. If done on purpose you >>>>win every game :-) >>>> >>>>>4. A program (ie chessmaster) might poll for input, consuming 1/2 of the cpu >>>>>even though it is not 'thinking'. >>>> >>>>Another true point. Every program needs to poll for input. Asking the >>>>keyboard / mouse for input are expensive (slow) operations. In Rebel >>>>I have a counter that makes sure to look for input after 500 evaluations. >>>>If I decrease the value to say 50 or 10 the NPS of Rebel drops >>>>tremendously (forgot about the exact slow-down). >>>> >>>>Thus the opponent program will ALWAYS slow-down yours and you can >>>>only guess how much that is. >>>> >>>>>There are _too_ many things a program _might_ do. I'll bet not one person >>>>>gave any thought to a "learning cycle" in crafty, yet it does it in every >>>>>game. And it steals 2-4 seconds of time from the opponent. In short time >>>>>controls, that might be important. >>>>> >>>>>If I _know_ people are testing like this, I'll bet I can raise Crafty's rating >>>>>by 100 points minimum. I won't say how, but it shouldn't take too much >>>>>imagination to figure it out. :) And with that said, why bother testing in a >>>>>way that is obviously potentially unreliable. For fun, sure. But reporting >>>>>the results as "A beats B" is not very scientific... A might not actually >>>>>be able to beat B, he just might have a smarter programmer that takes >>>>>advantage of a flawed testing methodology... >>>>> >>>>>:) >>>>> >>>>>Bob >>>> >>>>Strong points Bob. Still people are in love with the system as you only >>>>need one PC and have a lot of fun. If only its results are judged in the >>>>way it should. And in no way you can compare eng-eng matches on 1 PC with >>>>eng-eng matches on 2 PC's. >>>> >>>>Ed >>>> >>>>PS, match score sofar PB_ON vs PB_OFF 17.5 - 8.5 >>> >>> >>>I suspect you are going to get a 100+ point difference. When I first started >>>running Crafty on ICC, I was experiencing the usual "scanf()" buffering issue >>>with winboard and couldn't get pondering to work right, although it was fine >>>in text mode without xboard. When I got it right, crafty's rating instantly >>>jumped by about 120 rating points, maybe more. When I would turn it off, down >>>it would go by the same amount. I have, as a result, generally credited >>>ponder=on with roughly 120 rating points. Be interesting to see how your >>>results end. You are at about the 120 point level yourself, so far, if my >>>mental calculations are anywhere near right, winning 2 of every 3 games. 3 of >>>every 4 gets to +200, so maybe you are closer to 160 than 120... we will see... >> >> >>It's worth pointing out that whatever rating difference he gets is going to be >>inflated, since he is using the same engine on both machines and therefore >>getting perfect ponder accuracy. >> >>--Peter > > >It won't get 'perfect ponder accuracy' by a long shot. I doubt he will >predict over 50%. Because one engine is _always_ able to outsearch the other >by 1 ply on its turn. because both will be doing about the same depth >normally, but when one does an 8 ply search, it only has info based on a 7 ply >search by its opponent. When its opponent actually moves it will have 8 plies >(or whatever) to choose a move. > >Pondering accuracy won't be nearly as high as you think... particularly when >the game starts to turn tactical. It won't get perfect ponder accuracy but I believe that it will get more than 50% if you play the same engine against itself. I think the probability to change your mind in the last ply is clearly less than 50% and even more than 60%. Is there an evidence that I am wrong? When the game starts to be tactical you can sometimes get even more accuracy because the moves of the sides are sometimes forced. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.