Author: Dann Corbit
Date: 21:05:52 07/01/02
Go up one level in this thread
On July 01, 2002 at 23:46:54, pavel wrote: [snip] >stupid question: >how many games will it take (theoritically) to get 0 error margin? An infinite number. >or atleast 5-10 error margin with elostat. Depends on a lot of factors. How well the strength of the opponents is known (for instance). Certainly many tens of thousands of games, total to get a figure that good. But maybe a lot more than that. For instance, if you have only two programs or if the strength of the programs is either very close or very extreme, etc. The number of games verses ELO accuracy from this list: http://home.swipnet.se/~w-36794/ssdf/nr000.htm should be helpful for getting an idea. SWAG: 50 programs like those on the SSDF do a round robin with 10,000 games against each opponent. The interesting size for the error bar is when you can say that "program A is probably better than program B, because x-bar(A) - x-bar(B) is less than stdev" Considering the #1 program and the #10 program from the SSDF: 1 Deep Fritz 256MB Athlon 1200 MHz 2726 55 -51 182 65% 2618 10 Junior 7.0 128MB K6-2 450 MHz 2624 45 -42 271 65% 2518 2726-51=2675 2624+42=2666 Therefore we can say that Deep Fritz on 1200 MHz Athlon (under the conditions of the trials) is probably stronger than Junior 7 on a 450 MHz K6-2, with a pretty good measure of certainty.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.