Author: Severi Salminen
Date: 01:13:17 12/19/00
Go up one level in this thread
>>That begins to sound interesting. 200 games match still has some error margins >>but we'll see a lot from that result. I'm looking forward for the results - not >>too often someone runs a 200+ match here in CCC, thanks! >> >>Severi > > > >On 200 games, the margin of error for 80% reliability is +/-3.5%. >For 70% reliability it's +/-3.0%. > >If a program wins the 200 games match by 53.5% (107-93) or more, you can say >with 80% relability that it is stronger than its opponent. > >If it wins by only 53% (106-94) you can say it is better, but only with 70% >reliability. > >You see that when the programs are very close you need a very large number of >games to determine which is the best. Yes, that's what I also stated to Jorge. It would be great to have some table online (here at CCC) showing these error margins. Then it would be easier to convince people that if Fritz beats Crafty 4-0 it probably doesn't tell the whole truth... >On the other hand, if there is a significant difference before you reach 200 >games, it is possible to say which is the best without playing the 200 games. I wouldn't stop unless the difference is _very_ big. 20-0 is not enough because the small number of games played. It is hard to say what is the significant difference because that depends on number of games (like the reliability of the match). But of course if one program is leading 150-0 in a 200 games match one could end it saying A won at least 150-50. Here we have probably a quite big difference in strenght. Severi
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.