Author: blass uri
Date: 08:38:07 10/07/99
Go up one level in this thread
On October 07, 1999 at 10:13:13, Robert Hyatt wrote: >On October 07, 1999 at 00:08:13, Christophe Theron wrote: > >>On October 06, 1999 at 23:01:13, Robert Hyatt wrote: >> >>>On October 06, 1999 at 19:24:22, Christophe Theron wrote: >>> >>>>On October 06, 1999 at 18:44:51, Robert Hyatt wrote: >>>> >>>>>On October 06, 1999 at 17:36:43, Christophe Theron wrote: >>>>> >>>>>>On October 06, 1999 at 16:38:48, blass uri wrote: >>>>>> >>>>>>>On October 06, 1999 at 15:30:48, Robert Hyatt wrote: >>>>>>> >>>>>>> >>>>>>><snipped> >>>>>>>>4. A program (ie chessmaster) might poll for input, consuming 1/2 of the cpu >>>>>>>>even though it is not 'thinking'. >>>>>>> >>>>>>>This is easy to check and if the testers are intelligent they will not let it. >>>>>>> >>>>>>>I remember from previous posts some monthes ago that it is not a problem in >>>>>>>Didzis Cirulis's games because he gives the opponents of chessmaster more time >>>>>>>becuase of this problem. >>>>>> >>>>>> >>>>>>As far as I know only CM has this problem. >>>>>> >>>>>>Didzis has found a way to avoid it: after CM moves, it is enough to open a pull >>>>>>down menu to stop CM's polling input. >>>>>> >>>>>>This has been carefully verified, and it works. >>>>>> >>>>>> >>>>>> Christophe >>>>> >>>>> >>>>>Don't you think that is a problem? IE the pull-down is _always_ done instantly? >>>>>Never forgotten? I have played lots of manual games over the years. I have >>>>>missed an opponent's move from time to time... I can only imagine how easy it >>>>>is to bump the mouse and dismiss the pull-down without realizing after a long >>>>>game... >>>> >>>>Never unplugged your computer by accident? >>>> >>>>Never had a power failure? >>>> >>>>In this case your PB was spoiled! >>>> >>>>Damned! :) >>>> >>>> >>>> Christophe >>> >>> >>>I'm just trying to point out how many difficulties there are in one-computer >>>testing. I have always been a scientist, and like 'controlled experiments'. I >>>don't want to fall into the 'cold fusion' trap and look like an idiot. As a >>>result, I try to be _certain_ that any experiment I run has exactly one degree >>>of freedom, not two, or three, or an unknown number. Because that wrecks the >>>accuracy and reliability of the experiment being done. >> >> >>Of course. >> >> >> >>>In one computer testing there is no way to be sure that A doesn't interfere >>>with B, even though A is not 'pondering'. Fast games make it even worse. But >>>any such testing can produce different results than normal two-computer testing. >> >> >>There could be problem as "the opponent does some computation just after his >>move" or such, but you could trust the testers a little bit more. They are able >>to detect this and >>* either to find a way to avoid it >>* or to not play single computer matches with this program. >> >> >>Also, in the case of DOS programs, there is NO problem. You can set a propertiy >>of the DOS boxes under Windows, so they are completely frozen as soon as they >>are not in the foreground anymore. When you press Alt-Tab to switch to the other >>program, it is guaranteed that the first one receives 0% of the CPU usage. >> >> >> >> Christophe > >Remember your earlier comment about how everyone is testing crafty like this? >Crafty does that extra work _after_ making a move. How many out there have >discovered this? _ZERO_. Yet it has been doing this since learning was first >written. And not a person has mentioned the effect, even after playing hundreds >of game/1 minute type events... :) > >Here are the current 'degrees of freedom' in a typical chess match: > >1. two different programs >2. no pondering which affects timing allocation decisions >3. unknown side-effects like the 'learning cycle' above >4. using a non-native book for an engine (all engines use the fritz book, > for example). >5. tablebases/non-tablebase/some tablebases/sometimes tablebases only accessed > from the GUI vs the engine, etc. > >Here are the usual degrees of freedome in a normal chess match: > >1. two different programs. >2. (on rare occasions) different hardware speeds. > >somehow (a) and (b) look different to me. :) > >I am sure there are other differences that can be put under (a) like with two >engines, one machine, hash has to be smaller, which affects some programs more >than the other, etc. > >It is just a flawed experiment... I think that people are more interested in slower games and not in game/1 minute type events. I think that the original tester of this long thread Dizdis cirulis played clearly more slower games and manually. Of course the experiment is not the same as with 2 computers but I think that the difference is usually minor(less than 20 elo). I believe that Christophe Theron may improve his program by tuning to permanent brain on(he admitted that he does not do it) but I do not think that the improvement is going to be more than 20 elo. You also can improve your program for permanent brain off games by tuning it to this kind of game but I do not believe that you will get more than 20 elo realtive to a good guess without tuning. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.