Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: how not to calculate performance

Author: James T. Walker

Date: 04:42:24 11/02/04

Go up one level in this thread


On November 01, 2004 at 14:42:26, Uri Blass wrote:

>On October 31, 2004 at 11:32:11, James T. Walker wrote:
>
>>On October 31, 2004 at 07:11:04, Sune Fischer wrote:
>>
>>>On October 31, 2004 at 00:40:32, James T. Walker wrote:
>>>
>>>>On October 30, 2004 at 23:45:49, Sune Fischer wrote:
>>>>
>>>>>On October 30, 2004 at 21:59:36, James T. Walker wrote:
>>>>>>
>>>>>>As long as you realize you are making a "best guess" and not giving a real
>>>>>>rating that's fine.
>>>>>
>>>>>It was of course a back of the envelope, I have not derived it rigorously.
>>>>>
>>>>>I think a more accurate guess can be made if you solve for the case where the
>>>>>binomial distribution should give 50% or more for the X straight wins.
>>>>>
>>>>>> The problem is that in real life untill you actually score
>>>>>>some real points you cannot get a score which is anything but a guess.
>>>>>
>>>>>It will always be a guess as long as all you have is a finite sample.
>>>>
>>>>Well I'm talking about reality not theory.
>>>
>>>I don't understand why you make that distinction. The "real" rating is build
>>>upon the theorical model which we are discussing, so in essence there is no
>>>"real" rating there is only what the theory pridicts.
>>>
>>>I'm not sure model is good out in the tails, but that's a different story.
>>>
>>>>If you play 4 games and score 0
>>>>points vs players averaging 1400 your provisional rating will be 1000.  At that
>>>>point it's not a guess it's your actual rating which you take into your next
>>>>tournament.  NOBODY claims it's an exact rating which follows you through all
>>>>the days of your life.  This same formula is used to provide a "Performance
>>>>rating" in a match/tournament.
>>>
>>>The 400 is just a lazy mans approximation, for practical reasons people don't
>>>run around with calculators and use the exact formula, they often lookup the
>>>result on a small printed table.
>>>If you lose too much you end up outside the range of this table and they just
>>>subtract the 400.
>>>
>>>> It's again not just a guess it's how you
>>>>performed in that particular match/tournament.  Again this is not your actual
>>>>rating that you carry with you but simply an attempt to measure how you did in
>>>>one particular match/tournament.  But after you have played in some pre-defined
>>>>number of games you get an "established" rating.  Of course you know all this
>>>>but you want to quote some mathmatical theory that says that 0/4 is stronger
>>>>than 0/12.
>>>
>>>Losing 12 times in a row is worse than losing only 4 times, isn't this logical?
>>>
>>>If I play 4 grandmasters I will lose 4 times, does this mean I'm rated 2100?
>>>It only means that I'm probably rated below 2100, we cannot say much more than
>>>that.
>>>If I play 12 times against them and loses them all, then we can say that I'm
>>>probably 1800 or below.
>>>
>>>So you get more and more information with each game, one can say that the
>>>measurable range slowly extends out from the 2500 and it will eventually reach
>>>you.
>>>
>>>>I am saying you can't prove it untill some more data is acquired
>>>>which will separate  the 0/4 player from the 0/12 player.  I'ts like saying
>>>>0/999 is stronger than 0/1000.  Prove it !
>>>
>>>I guess one can say that it is "proven" that the 0/1000 guy is weaker than X,
>>>where for the 0/999 it is only "proven" that he is weaker than X+epsilon.
>>>
>>>The estimated score must be slightly higher for the 0/999 guy, as it has not
>>>been proven he is bad enough to lose all 1000.
>>
>>That is an invalid assumption on which you base your entire argument.  When in
>>fact if you have lost 999 games in a row the odds are that you will lose the
>>next game also.
>
>Yes,but you still cannot be sure about it so getting the information is
>relevant.
>
>You may be sure only with 94.99% that he is weaker than X after 999 games when
>you are sure in 95% that he is weaker than X after 1000 games.
>
>If you want to build some interval for his rating when you can be sure with 9 5%
>confidence that his rating is inside the interval then the interval may be
>(0,X) with 1000 games and something like (0,X+0.0001) with 999 games.
>
>Uri

Just plot a graph with 999 data points all with a value of zero.  Then plot
another graph of 1000 data points  all with a value of zero.  Then compare the
difference.
Jim



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.