Author: Robert Hyatt
Date: 18:27:56 05/19/00
Go up one level in this thread
On May 19, 2000 at 12:28:46, blass uri wrote: >On May 19, 2000 at 12:12:02, Ed Schröder wrote: > >>On May 19, 2000 at 10:27:04, blass uri wrote: >> >>>On May 19, 2000 at 09:42:07, Enrique Irazoqui wrote: >>> >>>>On May 19, 2000 at 09:37:19, Chris Carson wrote: >>>> >>>>>I am planning to publish an updated list list here with >>>>>all rated human vs computer results for 40/2 events. >>>>> >>>>>Please let me know your thoughts on the following: >>>>> >>>>>1. Exclude Performance Rating when 3 or fewer games >>>>> have been played by a program/hardware. >>>> >>>>I don't see why. >>>> >>>>>2. Exclude forfiets and protest resignations (Dutch Championship), >>>>> and games where computers lost due to hardware, IP failures, >>>>> or operator error. >>>> >>>>I would definitely exclude forfeits and IP failures, but not the rest. In my >>>>opinion, this list is interesting if it reflects the real performance of >>>>programs in actual games. Hardware failures and operator's errors are part of >>>>how a program plays. Forfeits and IP failures are not. >>>> >>>>Enrique >>> >>>Do you really think that losing on time is part of how shredder4 plays? >>> >>>I do not agree. >>>I think that operator's error are not part of how a program plays and it is not >>>fair to include the game that shredder lost on time in a winning position when >>>the reason was not a bug in the program. >> >>Then add the game Rebel-Shredder, Rebel-Hoffman etc, etc to the exception list. >>The list will soon become endless. Forfeits and IP failures are exceptions which >>are okay IMO. >> >>Rebel-Hoffman: hardware failure -> counts. >>Rebel-Shredder: operator troubles -> counts. > >This was not a computer-human game and the list was about computer-human games. > >>Shredder lost on time -> counts. >>Rebel-GM Ralf Akesson lost on time -> counts. >> >>Ed > >I suggest to do two lists(one with the exception and another list without the >exceptions). > >I think that the list without counting the exceptions gives a better picture >about the level of programs. > >Uri I think excluding hardware/operator problems gives an inflated view of the program's performance. Because in a real tournament, it _is_ going to have an occasional operator gaffe, and it _is_ going to have a hardware or communication failure. Had you attended any of the ACM events, you would have seen some program having a problem in _every_ round... If you lose one of twenty games due to a hardware crash, that is part of your performance ability... you are simply going to lose one of every twenty. If you exclude that, your tournament TPR will _always_ be lower than your "fake" rating because the "fake" rating doesn't include things that happen as a matter of course...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.