Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Rebel Performance Rating

Author: Stephen A. Boak

Date: 15:43:02 09/19/99

Go up one level in this thread


On September 19, 1999 at 18:22:45, odell hall wrote:


>   Personally, I think everyone is "pulling the Trigger" alittle to fast
>on this GM Question. (I hope I am not offending anyone with my "Pulling the
>Trigger statement) I think there are way to few games to jump to such
>Extravagant conclusions.(I mean with the rebel GM challenge). Can We truthfully
>believe that based on ten games, the whole grandmaster question is settled?
>Remember the 2480 number is including the loss to hoffman, which has been shown
>to be "suspect". Another Valid Question to consider is  "Is it Fair or
>reasonable to suggest that because one program achieved a certain ratings then
>they all would achieve the Same?  We are all willing to acknowledge that
>Programs all have different styles of play, would it be unreasonable to expect
>that they would also have different results? The Diversity in style between
>programs is as great as the diversity of play in humans.  Would anyone ever
>suggest that because "john doe" losses to X, then all humans would lose to X?
>I think overall everyone is  dissapointed in Rebel's performance, expecting
>rebel to have some win's against the GM's. (myself included) Our "expectations"
>not being met we tend to Exaggerate the significance of the results. As Dr Hyatt
>pointed out "2480" elo pointed out is very close To GM. Apparently not even Dr.
>Hyatt has ruled out the possibility that rebel will ultimately achieve it's
>goal.   Personally I don't agree with the whole format for determining GM
>strength for Computers, When did we take a vote and decide that only Rebel's
>results would determine for all computers whatever the GM Question is Answered?
>Why not take the Collective results of "All" the programs as a whole? This 2480
>is only meaningful if you decide to conviently exclude all the other results of
>computers against GM's.
>

Rebel is the only team I know of that is openly (publicly) trying to gauge their
software's performance against strong humans at tournament time controls with a
rigorous series of games.

I give them applause for testing their program openly for all to see the
results.

If Rebel succeeds in establishing the true strength of their program (perhaps
even improving it along the way) against humans under these settings, I would
not extrapolate the Rebel results to include other programs that have not openly
tested in such a rigorous way (with significant prizes for defeating and drawing
the program).

--Steve Boak



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.