Author: Dann Corbit
Date: 11:31:08 12/01/99
Go up one level in this thread
On November 30, 1999 at 22:14:34, David Blackman wrote: >On November 30, 1999 at 05:35:08, Dann Corbit wrote: >[ cut ] >>You make an excellent point and I will even concede that the method works. >>However, the problems that I mention (top players facing each other etc.) still >>hold. Further, what is the precise number of rounds required for an accurate >>rating? I think that is very much in doubt, especially if the seeding is really >>random. Depending upon the "badness" of the seeding, a much greater number of >>rounds could be required. > >Don't get too worried about top players meeting early. They have to meet >sometime. With a accurately seeded Swiss, it usually starts happening round 4 or >5. With a random Swiss it can happen just about anytime. > >Try running Swiss tournaments for a game where the higher rated player always >wins. If you simulate this on a computer, you should be able to run a large >tournament in about a second (but it takes longer to understand the results :-) >You'll find 6 rounds is enough for 50 players if you seed correctly. With random >seeding 6 rounds is still enough! > >The trouble starts if you have a more complicated game, like chess, where draws >are possible, and even the lower rated player can win occasionally. In that case >you need more rounds, and the seeded Swiss is a bit more efficient than the >random one. > >The number of rounds you need to be really sure is enormous however. A round >robin is not enough! The SSDF plays hundreds of games per program, and still >they are not sure who is really number 1. Usually with the uncertainties they >quote it could be any of half a dozen programs. Very true, and really no matter how hard you try you can never be positive of the best program even with SSDF and a million games per program. However, I think something that is going to proclaim "champion" ought to make an effort to get the best possible accuracy within allowed time constraints. The good thing about round-robin is that everyone plays everyone. I think with computer programs, some protocol could be developed (similar to the icky rs232 protocol but over tcp/ip or something reasonable) where the programs could play automatically, day and night, with the server automatically switching contestants. That way, lots of games could be played and the programmers could run around and have fun when the games were underway if they felt like it. You could have games running all night and all day. In a scenario like this, you could have the games be 40/2 + 1hour and run 8 games per machine pair per day. With (perhaps) 20 pairs of machines you could run 160 games in one day and 320 over a weekend. You could give the programmers beepers in case their program crashes or whatever and intervention is needed. I think that would answer all objections and produce a much better estimate of what the best program is. If each program just plays log(n) games, where n is the number of entrants, then the contest is really something of a joke as far as who the top program is [in ability]. On the other hand, perhaps the randomness is an allure to the crappy programs because even they have a chance in such a format.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.