Author: Ed Schröder
Date: 10:29:05 05/19/00
Go up one level in this thread
On May 19, 2000 at 13:26:26, Robert Hyatt wrote: >On May 19, 2000 at 12:41:57, Enrique Irazoqui wrote: > >>On May 19, 2000 at 12:22:05, Ed Schröder wrote: >> >>>On May 19, 2000 at 11:05:45, Enrique Irazoqui wrote: >>> >>>>On May 19, 2000 at 10:58:57, Robert Hyatt wrote: >>>> >>>>>On May 19, 2000 at 10:27:04, blass uri wrote: >>>>> >>>>>>On May 19, 2000 at 09:42:07, Enrique Irazoqui wrote: >>>>>> >>>>>>>On May 19, 2000 at 09:37:19, Chris Carson wrote: >>>>>>> >>>>>>>>I am planning to publish an updated list list here with >>>>>>>>all rated human vs computer results for 40/2 events. >>>>>>>> >>>>>>>>Please let me know your thoughts on the following: >>>>>>>> >>>>>>>>1. Exclude Performance Rating when 3 or fewer games >>>>>>>> have been played by a program/hardware. >>>>>>> >>>>>>>I don't see why. >>>>>>> >>>>>>>>2. Exclude forfiets and protest resignations (Dutch Championship), >>>>>>>> and games where computers lost due to hardware, IP failures, >>>>>>>> or operator error. >>>>>>> >>>>>>>I would definitely exclude forfeits and IP failures, but not the rest. In my >>>>>>>opinion, this list is interesting if it reflects the real performance of >>>>>>>programs in actual games. Hardware failures and operator's errors are part of >>>>>>>how a program plays. Forfeits and IP failures are not. >>>>>>> >>>>>>>Enrique >>>>>> >>>>>>Do you really think that losing on time is part of how shredder4 plays? >>>>>> >>>>>>I do not agree. >>>>>>I think that operator's error are not part of how a program plays and it is not >>>>>>fair to include the game that shredder lost on time in a winning position when >>>>>>the reason was not a bug in the program. >>>>>> >>>>>>Uri >>>>> >>>>> >>>>>Depends on your definition of "How Shredder plays". If you mean how it plays >>>>>in human events, then the answer is "yes". Because the operator _will_ make a >>>>>mistake here and there. Resigning when there is a deep saving move that the >>>>>program might have played without understanding it. Losing time on the clock >>>>>by going to the bathroom. Etc. The human operator _is_ part of the "system" >>>>>until we start using robots controlled by the computer. >>>>> >>>>>I have made mistakes (as an operator) that ending up costing Cray Blitz a game >>>>>here and there. In the WMCCC event in Jakarta, the operator misunderstood how >>>>>to set the time control and set it for 40 moves in 2 days, not 40 moves in 2 >>>>>hours. We lost the first game that way. If you have a human in the loop, then >>>>>he has to be factored in. As does hardware failures which _do_ happen in games. >>>>> >>>>>In fact, bleeding edge hardware is dangerous to use for this reason. >>>> >>>>This was my first reaction too, but I remember reading here that the operator of >>>>Shredder in the last round of the Israeli league lost on time almost on purpose, >>>>making telephone calls, not caring about the program, etc. So it is an >>>>exceptional case that in my opinion makes the game irrelevant for rating >>>>purposes. >>> >>>I understand the point you are making. The very same thing happened in >>>the 2 games Rebel8 played against GM Ralf Akesson. Rebel8 won the first >>>game and lost the second game on time due to an operator error in a >>>promising position. Make an exception? No way IMO. The next thing a GM >>>loses on time in a won position because his wife gave birth and he went >>>home. The list of exceptions soon becomes endless. We need a clear rule. >> >>Sure, but if the purpose of this rating list is to give us an idea of the >>strength of programs, I would discard games that we know are meaningless, like >>the 2 forfeits of Fritz in Holland and this Shredder game. The key word, to me, >>is "meaning", and this game has none. The list may be more complicated, but also >>more accurate. >> >>Enrique >> >>>Ed >>> >>> >>>>Enrique > > >Forfeits are "non-events" since they weren't played (even the 4 mover was a >non-game for obvious reasons). But operator errors are part of the computer >"system". And you can't always have a "perfect" operator. The best solution >is that the author is the _only_ one that operates. As he is the least likely >to make an error that influences the game outcome. But as soon as you use other >operators, the probability of error increases. And as it increases, the chances >that the program will perform somewhat below "expectation" become greater. > >But that is part of the "system" IMHO. Otherwise you start with "It lost that >game due to a power failure that lost some pondering time." "It lost this game >due to a hardware glitch that made me reboot and restart, losing information." >"It lost that game due to an operator typo that made it have to back up and lose >stuff." "It lost that game because the hardware crashed and wouldn't come back >up." > >Etc. > >All of those are part of computer chess. The human has his own set of problems >to contend with, and he can't escape them either. Totally in agreement. Ed
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.