Author: Severi Salminen
Date: 05:55:48 08/11/00
Go up one level in this thread
Hi! >I wonder, if it's possible to arrange match between the two top rated (against >humans) programs: Deep Junior 2702 (8*700Mhz) and PConners 2663 (160*300Mhz)? >Mini match with 4-10 games will give interesting info about their strength! 10 games is not enough. If you have two programs of equal strenght and play 10 games, you still have 75% chance that the result is not 5-5. I did the maths in Winboard forum yesterday (Topic: The funny thing about chessengines/Example 2): [snip]-------------------------------- Example: we have two programs (A and B) of equal strenght playing chess. P(A(1))=0.5 (A(1) means prog A wins 1 match). P(A(10))=0.5^10, right? P(A(6))=0.5^6*0.5^4*((10!/4!)/6!)=0.20=20%. I am not sure if the combination part was correct but that's not too important. P(A(5))=0.5^5*0.5^5*((10!/5!)/5!)=0.25=25%. P(A(7))=0.11=11%. So, we have 25% chance that these equally strong programs both actually score 5/10 in 10 game match. That is quite low probability in my opinion. And we have 40% change that the other scores 6/10 and the other 4/10. And still a 22% chance that the other scores 3/10 and the other 7/10. We have 75% chance that the result is _not_ correct!! When N is 10 the probability that the result is within (say) 20% window (that is: 4/10, 5/10 or 6/10) is 0.25+0.4=65%. When N grows these results tend to pack close to the expected result. When we have 20 games this same probability grows to 74%. I don't have any decent calculator with me, so I can't check how big N should be to get result with 95% of even 99% accuracy. And when we make the window more narrow, this probability decreases. [snip]-------------------------- So, IMHO, the probability to get false results is too high to consider playing less than 10 games. Severi
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.