Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Rating swings on ICC

Author: Peter Fendrich

Date: 02:08:41 08/03/98

Go up one level in this thread


On August 02, 1998 at 16:09:33, Don Dailey wrote:

>On August 02, 1998 at 15:36:35, Robert Hyatt wrote:
>
>>On August 02, 1998 at 14:46:34, Don Dailey wrote:
>>
>>>On August 02, 1998 at 13:00:56, Robert Hyatt wrote:
>>>
>>>>On August 02, 1998 at 09:37:24, Don Dailey wrote:
>>>>
>>>>>On August 02, 1998 at 08:01:28, Robert Hyatt wrote:
>>>>>
>>>>>>On July 31, 1998 at 20:51:56, Bruce Moreland wrote:
>>>>>>
>>>>>>>
>>>>>>>On July 31, 1998 at 14:00:04, Peter Fendrich wrote:
>>>>>>>
>>>>>>>>The ratings on ICC and FICS are really swinging. IMHO they are using a
>>>>>>>>completely wrong approach to handle this. The ELO formula is not at all suited
>>>>>>>>for the kind of events taking place on these real time severs. The original ELO
>>>>>>>>formula is using a kind of constant a 'dampening' factor for varitaions during
>>>>>>>>time. At least on ICC they have tried to use another, more dynamic but similar
>>>>>>>>method to handle this problem. There are new, much better, ways to deal with
>>>>>>>>this. Nowerdays used within some of the space, aero and automotive applications.
>>>>>>>
>>>>>>>Both ICC and FICS tried to use the Glicko system.
>>>>>>>
>>>>>>>On ICC, they made it an "extra" rating system, but I don't think anyone paid
>>>>>>>much attention to this, so it is removed from the "finger" notes.  It is still
>>>>>>>there, but to see it you have to use "yfinger".
>>>>>>>
>>>>>>>On FICS, they made this the only rating system.  I am not there much, but I
>>>>>>>remember seeing a lot of posts about this, people were upset about this system
>>>>>>>because if you played a lot, your rating tended to stay constant no matter how
>>>>>>>you did, and people wanted to have more movement in their ratings.  So I believe
>>>>>>>that on FICS they patched Glicko somehow, so that ratings would still move a
>>>>>>>bit.
>>>>>>>
>>>>>>>bruce
>>>>>>
>>>>>>
>>>>>>the problem with "Elo" is that the "K" factor was statistically derived from
>>>>>>the typical number of rated games a person would reasonably play in a year, and
>>>>>>the max expected rating change of a person over that time frame.  It is totally
>>>>>>wrong for a server where (say) a program plays 20,000 games per year.  Because
>>>>>>we see the huge swings that result from this.  Chances are that if you are a
>>>>>>2,000 player today, you will be a 2,000 player in 6 months, regardless of how
>>>>>>many games you play,  So it would be difficult to pick a formula that is fair
>>>>>>to those playing a dozen games a year and to those playing thousands.
>>>>>
>>>>>I wonder why they don't give people the option to use a smaller
>>>>>K factor?
>>>>>
>>>>>- Don
>>>>
>>>>
>>>>You can't do that.. It would grossly distort ratings...  someone plays a group
>>>>of opponents and decides which he can beat regularly, and then adjusts K to
>>>>maximize his rating increase...  expect to lose?  small K.  there's already
>>>>plenty of rating abuse there.  :)
>>>
>>>I don't recommend that people have the option to change it whenever
>>>they feel like it, perhaps they simply are allowed an initial choice
>>>and given one chance to change their minds later.  Or maybe they
>>>are allowed to change after 6 months with a given one.
>>>
>>>But a better idea might be to start with a high K and have it adjust
>>>dynamically depending on how much you play.  It shouldn't get
>>>ridiculously high or low but should be limted in either direction to
>>>reasonble numbers.  I also advocate that if you play someone who is
>>>not well established,  your K factor should drop too for that one
>>>game, since the results should not be weighted too heavily.
>>>
>>>I know these things can get tricky so I don't know if this is a
>>>reasonable suggestion or not (not that anyone is listening.)
>>>
>>>- Don
>>
>>At one point in time, FICS was doing something like this... the "K" factor
>>is reduced by playing games, and increased by passing time.  If you play
>>frequently, your K stays low because it is very doubtful that someone's rating
>>would fluctuate very much over the span of a few hours or a few days.  If you
>>don't play frequently, your K stays "up" since it is possible tht your rating
>>could change over a period of weeks or months...
>>
>>It seemed to be quite good (to me) but a human would win 5 or 6 games in a
>>row against someone and complain when their rating did not jump like it would
>>in a USCF event.  The problem is everyone wants + changes, but no one wants big
>>- changes.  :)
>>
>>Bob
>
>Ok,
>
>One other suggestion.  Post 2 ratings, one with high K and another with
>low K.  Better yet, use only low K and publish a performance rating
>based on the last 20 games and call it "recent performance" or something
>like this.  Then players can see good results get immediately rewarded!
>
>- Don



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.