Author: Dann Corbit
Date: 14:49:52 08/24/01
Go up one level in this thread
On August 24, 2001 at 17:37:56, Jeff Lischer wrote: >On August 24, 2001 at 14:10:11, Dann Corbit wrote: > >> >>With the number of games played, the ELO figures are nearly MEANINGLESS. >> >>The error bars will mean that a program from amongh the weakest could really be >>the strongest. >> >>With +/- 200 ELO for each program (even with one standard deviation) you can see >>how the figures could easily be shaken up. >> >>You can calculate a TPR and all that. But the significance of the ELO figures >>is moot. > >Yes the Elo figures are almost meaningless because of the uncertainties, but I >don't think we are asking are these the correct Elo ratings. What we are trying >to figure out is what is the best measure of true performance at the tournament? >Are these Elo/TPR figures the best indication of performance or is there some >other measure that would be better? It seems to me that these Elo figures are a >better measure of performance than the raw score obtained, but I'm not sure >about that. The comment about how tournament strategy can affect the outcome is >a complicating factor. Tournaments like this do not show which program is the best. They show which program is the winner. If (before the contest) it was stated: "The program with the highest TPR ELO will be declared the winner." Then that program would be the winner. Since no such statement was made, it makes for fun discussion, but has no bearing on reality. You might as well use a random number generator, because the ELO figures do not demonstrate the point to be made because of their uncertainty. (IOW -- this program is stronger than that one because it has a higher TPR simply DOES NOT FOLLOW from the data provided) Tiny, short contests like this one do not [I REPEAT *DO* *NOT*] show which program is stronger than the others. They show which program won the official contest. That is all. It may be that the strongest program wins. In fact, odds are somewhat in the favor of the strongest program. However, it is possible (though unlikely) for the weakest program to win.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.