Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Rating in ICC is meaningless and here is an example

Author: Richard Pijl

Date: 11:28:54 01/15/03

Go up one level in this thread


On January 15, 2003 at 13:03:16, Miguel A. Ballicora wrote:

>On January 15, 2003 at 12:52:42, Robert Hyatt wrote:
>
>>On January 15, 2003 at 12:07:45, Richard Pijl wrote:
>>
>>>>
>>>>How can it be?  The order of the games is going to influence the rating
>>>>significantly since more recent games have more weight than earlier games.
>>>>
>>>That is not what he suggested.
>>>
>>>The difference in calculating is very small, but in extreme cases a very
>>>different result
>>>
>>>What Miguel suggested is to calculate the expected result for each game for an
>>>assumed rating, and then add all those expected results together. If the real
>>>result is higher than this sum, try again with a higher assumed rating, if it's
>>>lower, try again with a lower assumed rating until the best approximation is
>>>found.
>>>
>>>The advantage is quite clear when using an extreme example:
>>>
>>>Suppose you play:
>>>9 games against a 1000 player with 50%
>>>1 game against a 2000 player with 0 %
>>>
>>>score 45% with average opposition 1100 -> TPR just below 1100
>>>
>>>Let's say we play 5 more games against the 2000 player.
>>>Score is now 33%, average opposition 1400 -> TPR rises to somewhere around 1250
>>>if I'm not mistaken.
>>>
>>
>>As I have said, that is not a realistic happening for _normal_ rating
>>scenarios.  But on ICC it is likely and the problem is that we are now
>>rating _matches_ between a rated and unrated player.  Unfortunately there
>>is no real alternative other than to make new players play in a couple of
>>"rating tournaments" before playing individual players in matches, which is
>>doable and would have a better result than any attempt at fixing the
>>provisional scheme which is not broken.
>>

For a rating tournament you need rated people that want to participate in such
tournaments. That might be a bigger problem than fixing the provisional rating
scheme

>>
>>>Now compare with Miguels scheme.
>>>
>>>Start with the assumed rating of 1000:
>>>9x against 1000 -> expected result 4.5
>>>1x against 2000 -> expected result 0.01 (or something like that, very small)
>>>
>>>This is very close to the real result.
>>>
>>>Now the additional games:
>>>
>>>again assume a rating of 1000
>>>9x against 1000 -> exp. result 4.5
>>>6x against 2000 -> exp. result 0.06
>>>
>>>Again, quite close to the real result.
>>
>>Play the 2000 players first.  Now what??
>
>To be honest, I do not understand what you don't understand.
>
>4.5 + 0.06 = 0.06 + 4.5
>
>What is the relevance of what game was played first?

Well, odd enough it is relevant in ICC as you cannot match a player which whom
you have a bigger rating difference than 400 IIRC.
Which makes the problem of the current scheme even worse. When you start of at
ICC, and lose your first few games, it gets harder and harder to compensate as
you are not allowed to match the stronger players. That measure is of course
necessary to prevent someone from gaining rating points by losing to strong
players.
>
>>>As far as I know rating calculation for established ratings is done in a similar
>>>way to Miguels suggestion; not taking the average of the opponent's rating, but
>>>base it on the sum of expected results instead.
>>
>>How can you know the expected result _prior_ to having any results to base it
>>on?  That is what the provisional rating period tries to establish, and it does
>>so pretty well overall.

If you have no results at all it's just a guess. If its all wins or all losses
it's still a guess (although it's giving you an upper or lower bound). _Any_
system you pick can only work when the result is not zero points and not 100%.

If the spread of the rating of your opponents is very limited (within let's say
100 points) TPR works fine. In fact, the scheme that Miguel proposed will give
you almost the same result. But the farther apart the ratings of the opponents
are, the more TPR goes wrong. The rule that you cannot challenge opponents with
a rating difference > 400 is ICC's method of fixing this ...

Richard

>
>No it sucks, but the only reason why it is not more obvious is that, as you
>said, generally players face opponents that are relatively close to their
>ratings. It is not the case in ICC as you say, and it was not the case for
>Karpov when it played the US Amateur Team Championship in 98. In those cases, it
>shows.
>
>Miguel




This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.