Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Rating swings on ICC

Author: Don Dailey

Date: 13:09:33 08/02/98

Go up one level in this thread


On August 02, 1998 at 15:36:35, Robert Hyatt wrote:

>On August 02, 1998 at 14:46:34, Don Dailey wrote:
>
>>On August 02, 1998 at 13:00:56, Robert Hyatt wrote:
>>
>>>On August 02, 1998 at 09:37:24, Don Dailey wrote:
>>>
>>>>On August 02, 1998 at 08:01:28, Robert Hyatt wrote:
>>>>
>>>>>On July 31, 1998 at 20:51:56, Bruce Moreland wrote:
>>>>>
>>>>>>
>>>>>>On July 31, 1998 at 14:00:04, Peter Fendrich wrote:
>>>>>>
>>>>>>>The ratings on ICC and FICS are really swinging. IMHO they are using a
>>>>>>>completely wrong approach to handle this. The ELO formula is not at all suited
>>>>>>>for the kind of events taking place on these real time severs. The original ELO
>>>>>>>formula is using a kind of constant a 'dampening' factor for varitaions during
>>>>>>>time. At least on ICC they have tried to use another, more dynamic but similar
>>>>>>>method to handle this problem. There are new, much better, ways to deal with
>>>>>>>this. Nowerdays used within some of the space, aero and automotive applications.
>>>>>>
>>>>>>Both ICC and FICS tried to use the Glicko system.
>>>>>>
>>>>>>On ICC, they made it an "extra" rating system, but I don't think anyone paid
>>>>>>much attention to this, so it is removed from the "finger" notes.  It is still
>>>>>>there, but to see it you have to use "yfinger".
>>>>>>
>>>>>>On FICS, they made this the only rating system.  I am not there much, but I
>>>>>>remember seeing a lot of posts about this, people were upset about this system
>>>>>>because if you played a lot, your rating tended to stay constant no matter how
>>>>>>you did, and people wanted to have more movement in their ratings.  So I believe
>>>>>>that on FICS they patched Glicko somehow, so that ratings would still move a
>>>>>>bit.
>>>>>>
>>>>>>bruce
>>>>>
>>>>>
>>>>>the problem with "Elo" is that the "K" factor was statistically derived from
>>>>>the typical number of rated games a person would reasonably play in a year, and
>>>>>the max expected rating change of a person over that time frame.  It is totally
>>>>>wrong for a server where (say) a program plays 20,000 games per year.  Because
>>>>>we see the huge swings that result from this.  Chances are that if you are a
>>>>>2,000 player today, you will be a 2,000 player in 6 months, regardless of how
>>>>>many games you play,  So it would be difficult to pick a formula that is fair
>>>>>to those playing a dozen games a year and to those playing thousands.
>>>>
>>>>I wonder why they don't give people the option to use a smaller
>>>>K factor?
>>>>
>>>>- Don
>>>
>>>
>>>You can't do that.. It would grossly distort ratings...  someone plays a group
>>>of opponents and decides which he can beat regularly, and then adjusts K to
>>>maximize his rating increase...  expect to lose?  small K.  there's already
>>>plenty of rating abuse there.  :)
>>
>>I don't recommend that people have the option to change it whenever
>>they feel like it, perhaps they simply are allowed an initial choice
>>and given one chance to change their minds later.  Or maybe they
>>are allowed to change after 6 months with a given one.
>>
>>But a better idea might be to start with a high K and have it adjust
>>dynamically depending on how much you play.  It shouldn't get
>>ridiculously high or low but should be limted in either direction to
>>reasonble numbers.  I also advocate that if you play someone who is
>>not well established,  your K factor should drop too for that one
>>game, since the results should not be weighted too heavily.
>>
>>I know these things can get tricky so I don't know if this is a
>>reasonable suggestion or not (not that anyone is listening.)
>>
>>- Don
>
>At one point in time, FICS was doing something like this... the "K" factor
>is reduced by playing games, and increased by passing time.  If you play
>frequently, your K stays low because it is very doubtful that someone's rating
>would fluctuate very much over the span of a few hours or a few days.  If you
>don't play frequently, your K stays "up" since it is possible tht your rating
>could change over a period of weeks or months...
>
>It seemed to be quite good (to me) but a human would win 5 or 6 games in a
>row against someone and complain when their rating did not jump like it would
>in a USCF event.  The problem is everyone wants + changes, but no one wants big
>- changes.  :)
>
>Bob

Ok,

One other suggestion.  Post 2 ratings, one with high K and another with
low K.  Better yet, use only low K and publish a performance rating
based on the last 20 games and call it "recent performance" or something
like this.  Then players can see good results get immediately rewarded!

- Don









This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.