Author: blass uri
Date: 12:15:54 02/24/00
Go up one level in this thread
On February 24, 2000 at 13:34:35, Dann Corbit wrote: >On February 24, 2000 at 06:17:12, blass uri wrote: > >>On February 24, 2000 at 05:16:48, Dann Corbit wrote: >> >>>ELO is self adjusting. >>> >>>Programs that learn will rise in ELO. >>> >>>Programs that repeat the same mistakes will drop in ELO. >> >>This is not the problem. >>The problem is that programs that learn can get better rating if they play more >>games against program that do not learn. >> >>Suppose the ssdf plays 1000 games between Fritz5.32(p200) and shredder2(p200) >>that does not learn and Fritz5.32 wins 990:10 >> >>Fritz5.32(p200) is going to lead the ssdf list and Fritz5.32 on faster hardware >>is going to have lower rating only because you let Fritz5.32(p200) to play 1000 >>games against a program that does not learn when other programs played not more >>than 40 games. >> >>1000 games between 2 opponents does not happen but it does not change the fact >>that the there are errors in the rating because of the fact that the number of >>games against programs like shredder2(p200) is not exactly the same for all the >>programs. >> >>The errors are not big like my extreme example but there are errors. > >I don't think it is a problem, rather it is a correction. If (for instance) a >program can be beaten by the exact same line 100% the actual ELO of that program >is (or should be) zero. Aggressively learning programs against programs that do >not learn at all will eventually stomp them out of existance. The actual elo of programs that do not learn is practically not zero. Shredder2 has no learning function and the ssdf continue to test it. I do not know if shredder2 is deterministic in the middle game but even if it is not deterministic programs can get something close to 100% against it after enough games by repeating the same winning opening again and again. If the ssdf rating of shredder2 should be zero then there is a big problem because the rating is based on the assumption that shredder2 has rating close to 2500. If the rating of shredder2 should be 0 then the only possible solution to this problem is simply not to include programs without learning as players in the ssdf list and to delete all the results of games of programs with no learning function. It is possible to assume that the rating of shredder2 is not 0 because of the practical fact that there are finite number of games but in this case shredder2 has to play the same number of games against every opponent(it is possible to decide always to include only the first 40 games of shredder2 against every opponent and if program has less than 40 games against shredder2 simply not to include them for the calculation of the rating. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.