Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: how not to calculate performance

Author: Uri Blass

Date: 11:42:26 11/01/04

Go up one level in this thread


On October 31, 2004 at 11:32:11, James T. Walker wrote:

>On October 31, 2004 at 07:11:04, Sune Fischer wrote:
>
>>On October 31, 2004 at 00:40:32, James T. Walker wrote:
>>
>>>On October 30, 2004 at 23:45:49, Sune Fischer wrote:
>>>
>>>>On October 30, 2004 at 21:59:36, James T. Walker wrote:
>>>>>
>>>>>As long as you realize you are making a "best guess" and not giving a real
>>>>>rating that's fine.
>>>>
>>>>It was of course a back of the envelope, I have not derived it rigorously.
>>>>
>>>>I think a more accurate guess can be made if you solve for the case where the
>>>>binomial distribution should give 50% or more for the X straight wins.
>>>>
>>>>> The problem is that in real life untill you actually score
>>>>>some real points you cannot get a score which is anything but a guess.
>>>>
>>>>It will always be a guess as long as all you have is a finite sample.
>>>
>>>Well I'm talking about reality not theory.
>>
>>I don't understand why you make that distinction. The "real" rating is build
>>upon the theorical model which we are discussing, so in essence there is no
>>"real" rating there is only what the theory pridicts.
>>
>>I'm not sure model is good out in the tails, but that's a different story.
>>
>>>If you play 4 games and score 0
>>>points vs players averaging 1400 your provisional rating will be 1000.  At that
>>>point it's not a guess it's your actual rating which you take into your next
>>>tournament.  NOBODY claims it's an exact rating which follows you through all
>>>the days of your life.  This same formula is used to provide a "Performance
>>>rating" in a match/tournament.
>>
>>The 400 is just a lazy mans approximation, for practical reasons people don't
>>run around with calculators and use the exact formula, they often lookup the
>>result on a small printed table.
>>If you lose too much you end up outside the range of this table and they just
>>subtract the 400.
>>
>>> It's again not just a guess it's how you
>>>performed in that particular match/tournament.  Again this is not your actual
>>>rating that you carry with you but simply an attempt to measure how you did in
>>>one particular match/tournament.  But after you have played in some pre-defined
>>>number of games you get an "established" rating.  Of course you know all this
>>>but you want to quote some mathmatical theory that says that 0/4 is stronger
>>>than 0/12.
>>
>>Losing 12 times in a row is worse than losing only 4 times, isn't this logical?
>>
>>If I play 4 grandmasters I will lose 4 times, does this mean I'm rated 2100?
>>It only means that I'm probably rated below 2100, we cannot say much more than
>>that.
>>If I play 12 times against them and loses them all, then we can say that I'm
>>probably 1800 or below.
>>
>>So you get more and more information with each game, one can say that the
>>measurable range slowly extends out from the 2500 and it will eventually reach
>>you.
>>
>>>I am saying you can't prove it untill some more data is acquired
>>>which will separate  the 0/4 player from the 0/12 player.  I'ts like saying
>>>0/999 is stronger than 0/1000.  Prove it !
>>
>>I guess one can say that it is "proven" that the 0/1000 guy is weaker than X,
>>where for the 0/999 it is only "proven" that he is weaker than X+epsilon.
>>
>>The estimated score must be slightly higher for the 0/999 guy, as it has not
>>been proven he is bad enough to lose all 1000.
>
>That is an invalid assumption on which you base your entire argument.  When in
>fact if you have lost 999 games in a row the odds are that you will lose the
>next game also.

Yes,but you still cannot be sure about it so getting the information is
relevant.

You may be sure only with 94.99% that he is weaker than X after 999 games when
you are sure in 95% that he is weaker than X after 1000 games.

If you want to build some interval for his rating when you can be sure with 9 5%
confidence that his rating is inside the interval then the interval may be
(0,X) with 1000 games and something like (0,X+0.0001) with 999 games.

Uri



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.