Author: Dann Corbit
Date: 17:02:37 05/31/02
Go up one level in this thread
On May 31, 2002 at 19:22:27, Rolf Tueschen wrote: >On May 31, 2002 at 19:01:53, Dann Corbit wrote: > >>Since people are so often confused about it, it seems a good idea to write a >>FAQ. >>Rolf's questions could be added, and a search through the CCC archives could >>find some more. >> >>Certainly the games against the old opponents is always a puzzle to newcomers >>who do not understand why calibration against an opponent of precisely known >>strength is of great value. > > >No pun intended, but excuse me, you can't mean it this way! Are we caught in a >new circle? How can the older program be precisely known in its strength? >Of course it it isn't! Because it had the same status the new ones have today... > >And the all the answers from Bertil follow that same fallacious line. It's a >pity! > >Also, what is calibration in SSDF? Comparing the new unknown with the old >unknown? No pun inded. > >Before making such a FAQ let's please find some practical solutions for SSDF. The older programs have been carefully calibrated by playing many hundreds of games. Hence, their strength in relation to each other and to the other members of the pool is very precisely known. The best possible test you can make is to play an unknown program against the best known programs. This will arrive at an accurate ELO score faster than any other way. Programs that are evenly matched are not as good as programs that are somewhat mismatched. Programs that are terribly mismatched are not as good as programs that are somewhat mismatched. If I have two programs of exactly equal ability, it will take a huge number of games to get a good reading on their strength in relation to one another. On the other hand, if one program is 1000 ELO better than another, then one or two fluke wins will drastically skew the score. An ELO difference of 100 to 150 is probably just about ideal.
This page took 0.04 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.