Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: how not to calculate performance

Author: Stephen A. Boak

Date: 22:47:54 10/23/04

Go up one level in this thread


On October 23, 2004 at 18:47:26, Vincent Lejeune wrote:

>On October 23, 2004 at 16:37:37, Stephen A. Boak wrote:
>
>>On October 22, 2004 at 18:52:13, Uri Blass wrote:
>>
>>>On October 22, 2004 at 18:30:34, James T. Walker wrote:
>>>
>>>>On October 22, 2004 at 13:32:57, Uri Blass wrote:
>>>>
>>>>>go to the following link
>>>>>
>>>>>http://georgejohn.bcentralhost.com/TCA/perfrate.html
>>>>>
>>>>>enter 1400 for 12 opponents
>>>>>enter 0 for your total score
>>>>>
>>>>>Your performance is 1000 but if you enter 1 to your total score your performance
>>>>>is only 983.
>>>>>
>>>>>It seems that the program in that link assume that when the result is 100% or 0%
>>>>>your performance is 400 elo less that your weakest opponent but when your score
>>>>>is not 100% it has not that limit so they get illogical results.
>>>>>
>>>>>Uri
>>>>
>>>>My take on this is they are using a bad formula or have screwed up the program
>>>>to calculate the Rp.
>>>>The USCF uses Rp=Rc + 400(W-L)/N
>>>
>>>It seems that the USCF does not do it in that way
>>>
>>>They admit that the formula is not correct for players who won all their games
>>>
>>>Note:  In the case of a perfect or zero score the performance rating is
>>>estimated as either 400 points higher or lower, respectively, than the rating of
>>>highest or lowest rated opponent.
>>>
>>>It is probably better to estimate the preformance based on comparison to  the
>>>case that the player did almost perfect score.
>>>
>>>Uri
>>
>>Dear Uri,
>>What is the *correct* formula for a player who has won (or lost) all his games?
>>:)
>>Regards,
>>--Steve
>
>
>For such a player, the error margin = infinity
>
>the perf = average opp +400 to +infinity

Thanks, Vincent.  I know the formula well.  :)

I was poking fun at Uri (just teasing) for complaining about 'logic' when in
fact the formula for all wins or all losses is purely arbitrary.

[I've read that Uri is a mathematician, so I like to occasionally jump in and
comment when he seems to overlook something basic.  All in good fun--I
appreciate his postings and chess programming contributions.]

I asked Uri what formula would he suggest as 'correct'.

I don't think he could find a 'more logical' formula.  Arbitrary is arbitrary.
Any formula he might suggest would be just as 'illogical' (to use his word) as
the standard definition.

The fact is, when the thing to be measured via statistics is 'off the scale' or
'out of bounds', then the attempt to measure [especially when based on very few
samples] largely fails.

When the measuring stick is the wrong size for the object to be measured, then
the results yield little information.

Similarly, if the opponents' playing abilities are far above or below the
player's own abilities, using those opponents as the 'measuring stick' isn't
very helpful.  Statistical conclusions based on results against such non-equals
yields little information regarding the player's true rating level.

One can conclude stronger or weaker in a relative way ... but not an exact
rating or a rating within a reasonably acceptable margin of error or confidence
interval.

As you say, the margin of error would be infinite.

Under the circumstances of Uri's example, an estimate of 1000 (per standard
definition) or less than 983 (per Uri suggestion) are equally devoid of
precision.  Both are certainly 'substantially weaker' than the opponents'
ratings, and as such are equally valid (and equally arbritrary).

Uri would alter the 'scale' [i.e. standard definition, for all wins or all
losses] to achieve a 'logical' rating per his own arbitrary sense of 'logic'.

However, no matter what formula Uri could suggest, there are identical
situations (one could easily alter the ratings in his example to illustrate
those situations) in which his own formula would lead to the exact same
criticism he raises.

Hence some humor (irony), which I was trying to illustrate.

Regards,
--Steve



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.