Author: John Sidles
Date: 13:52:34 02/18/06
Go up one level in this thread
On February 18, 2006 at 03:50:09, Uri Blass wrote: >My question is based on your experience what is the biggest result that A beat B >in match of 100 games(Noomen match or match based on other positions like Albert >Silver's postions) but still A is not better than B against other programs. Here's a table for how large a plus score you need to see, by either A or B, for you to be confident (at the given level of confident) this plus score was not due to luck. Here "luck" means that A and B actually each have 1/3 chance of win, lose and draw, but that either program was simply lucky enough to achieve a plus score. nGame = 10, confidence = 95.%, score = +2.5 nGame = 10, confidence = 99.%, score = +3. nGame = 10, confidence = 99.9%, score = +4. nGame = 20, confidence = 95.%, score = +3.5 nGame = 20, confidence = 99.%, score = +4.5 nGame = 20, confidence = 99.9%, score = +6. nGame = 30, confidence = 95.%, score = +4.5 nGame = 30, confidence = 99.%, score = +5.5 nGame = 30, confidence = 99.9%, score = +7. nGame = 40, confidence = 95.%, score = +5. nGame = 40, confidence = 99.%, score = +6.5 nGame = 40, confidence = 99.9%, score = +8.5 nGame = 50, confidence = 95.%, score = +5.5 nGame = 50, confidence = 99.%, score = +7.5 nGame = 50, confidence = 99.9%, score = +9.5 nGame = 75, confidence = 95.%, score = +7. nGame = 75, confidence = 99.%, score = +9. nGame = 75, confidence = 99.9%, score = +11.5 nGame = 100, confidence = 95.%, score = +8. nGame = 100, confidence = 99.%, score = +10.5 nGame = 100, confidence = 99.9%, score = +13.5 So for example, in a 10 game tournament, if either program achieves a +4 score (or higher), you can be 99.9% confident that such a high score was *not* due to luck. Note: these were calculated by brute force in Mathematica.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.