Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Next Human vs Computer ratings list - I need opinions

Author: Robert Hyatt

Date: 07:58:57 05/19/00

Go up one level in this thread


On May 19, 2000 at 10:27:04, blass uri wrote:

>On May 19, 2000 at 09:42:07, Enrique Irazoqui wrote:
>
>>On May 19, 2000 at 09:37:19, Chris Carson wrote:
>>
>>>I am planning to publish an updated list list here with
>>>all rated human vs computer results for 40/2 events.
>>>
>>>Please let me know your thoughts on the following:
>>>
>>>1.  Exclude Performance Rating when 3 or fewer games
>>>    have been played by a program/hardware.
>>
>>I don't see why.
>>
>>>2.  Exclude forfiets and protest resignations (Dutch Championship),
>>>    and games where computers lost due to hardware, IP failures,
>>>    or operator error.
>>
>>I would definitely exclude forfeits and IP failures, but not the rest. In my
>>opinion, this list is interesting if it reflects the real performance of
>>programs in actual games. Hardware failures and operator's errors are part of
>>how a program plays. Forfeits and IP failures are not.
>>
>>Enrique
>
>Do you really think that losing on time is part of how shredder4 plays?
>
>I do not agree.
>I think that operator's error are not part of how a program plays and it is not
>fair to include the game that shredder lost on time in a winning position when
>the reason was not a bug in the program.
>
>Uri


Depends on your definition of "How Shredder plays".  If you mean how it plays
in human events, then the answer is "yes".  Because the operator _will_ make a
mistake here and there.  Resigning when there is a deep saving move that the
program might have played without understanding it.  Losing time on the clock
by going to the bathroom.  Etc. The human operator _is_ part of the "system"
until we start using robots controlled by the computer.

I have made mistakes (as an operator) that ending up costing Cray Blitz a game
here and there.  In the WMCCC event in Jakarta, the operator misunderstood how
to set the time control and set it for 40 moves in 2 days, not 40 moves in 2
hours.  We lost the first game that way.  If you have a human in the loop, then
he has to be factored in.  As does hardware failures which _do_ happen in games.

In fact, bleeding edge hardware is dangerous to use for this reason.






This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.