Author: Robert Hyatt
Date: 07:13:13 10/07/99
Go up one level in this thread
On October 07, 1999 at 00:08:13, Christophe Theron wrote:
>On October 06, 1999 at 23:01:13, Robert Hyatt wrote:
>
>>On October 06, 1999 at 19:24:22, Christophe Theron wrote:
>>
>>>On October 06, 1999 at 18:44:51, Robert Hyatt wrote:
>>>
>>>>On October 06, 1999 at 17:36:43, Christophe Theron wrote:
>>>>
>>>>>On October 06, 1999 at 16:38:48, blass uri wrote:
>>>>>
>>>>>>On October 06, 1999 at 15:30:48, Robert Hyatt wrote:
>>>>>>
>>>>>>
>>>>>><snipped>
>>>>>>>4. A program (ie chessmaster) might poll for input, consuming 1/2 of the cpu
>>>>>>>even though it is not 'thinking'.
>>>>>>
>>>>>>This is easy to check and if the testers are intelligent they will not let it.
>>>>>>
>>>>>>I remember from previous posts some monthes ago that it is not a problem in
>>>>>>Didzis Cirulis's games because he gives the opponents of chessmaster more time
>>>>>>becuase of this problem.
>>>>>
>>>>>
>>>>>As far as I know only CM has this problem.
>>>>>
>>>>>Didzis has found a way to avoid it: after CM moves, it is enough to open a pull
>>>>>down menu to stop CM's polling input.
>>>>>
>>>>>This has been carefully verified, and it works.
>>>>>
>>>>>
>>>>> Christophe
>>>>
>>>>
>>>>Don't you think that is a problem? IE the pull-down is _always_ done instantly?
>>>>Never forgotten? I have played lots of manual games over the years. I have
>>>>missed an opponent's move from time to time... I can only imagine how easy it
>>>>is to bump the mouse and dismiss the pull-down without realizing after a long
>>>>game...
>>>
>>>Never unplugged your computer by accident?
>>>
>>>Never had a power failure?
>>>
>>>In this case your PB was spoiled!
>>>
>>>Damned! :)
>>>
>>>
>>> Christophe
>>
>>
>>I'm just trying to point out how many difficulties there are in one-computer
>>testing. I have always been a scientist, and like 'controlled experiments'. I
>>don't want to fall into the 'cold fusion' trap and look like an idiot. As a
>>result, I try to be _certain_ that any experiment I run has exactly one degree
>>of freedom, not two, or three, or an unknown number. Because that wrecks the
>>accuracy and reliability of the experiment being done.
>
>
>Of course.
>
>
>
>>In one computer testing there is no way to be sure that A doesn't interfere
>>with B, even though A is not 'pondering'. Fast games make it even worse. But
>>any such testing can produce different results than normal two-computer testing.
>
>
>There could be problem as "the opponent does some computation just after his
>move" or such, but you could trust the testers a little bit more. They are able
>to detect this and
>* either to find a way to avoid it
>* or to not play single computer matches with this program.
>
>
>Also, in the case of DOS programs, there is NO problem. You can set a propertiy
>of the DOS boxes under Windows, so they are completely frozen as soon as they
>are not in the foreground anymore. When you press Alt-Tab to switch to the other
>program, it is guaranteed that the first one receives 0% of the CPU usage.
>
>
>
> Christophe
Remember your earlier comment about how everyone is testing crafty like this?
Crafty does that extra work _after_ making a move. How many out there have
discovered this? _ZERO_. Yet it has been doing this since learning was first
written. And not a person has mentioned the effect, even after playing hundreds
of game/1 minute type events... :)
Here are the current 'degrees of freedom' in a typical chess match:
1. two different programs
2. no pondering which affects timing allocation decisions
3. unknown side-effects like the 'learning cycle' above
4. using a non-native book for an engine (all engines use the fritz book,
for example).
5. tablebases/non-tablebase/some tablebases/sometimes tablebases only accessed
from the GUI vs the engine, etc.
Here are the usual degrees of freedome in a normal chess match:
1. two different programs.
2. (on rare occasions) different hardware speeds.
somehow (a) and (b) look different to me. :)
I am sure there are other differences that can be put under (a) like with two
engines, one machine, hash has to be smaller, which affects some programs more
than the other, etc.
It is just a flawed experiment...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.