Author: blass uri
Date: 20:49:12 10/22/98
Go up one level in this thread
On October 22, 1998 at 21:07:00, Don Dailey wrote: >On October 22, 1998 at 20:41:48, Dann Corbit wrote: > >>On October 22, 1998 at 17:14:19, Howard Exner wrote: >>>Has anyone tried this ... Running a 20 game blitz match with the same >>>program but labeling the one program A and the other B. So the first >>>game would for example have Crafty A vs. Crafty B. Then the match would >>>continue with Crafty B vs. Crafty A and so on. >>> >>>What would be the results after a 20 game match? What if many 20 >>>game matches were run like this? Could this tell us something about >>>the predictability of what 20 game match scores tell us? Since the >>>programs competing are dead equal one would suspect a 10 - 10 result. >>>I'm curious what might be the range of the match results. >>> >>>I tried a 10 game match like this a while ago using Rebel 8. >>>The results of Rebel A vs Rebel B was 6.5-3.5. >>> >>>If someone has data on this could they post it. >>I think that shows, without a doubt that rebel A is much stronger than rebel B >>;-) [See my other rantings elsewhere]. >> >>I think this experiment is equally interesting to any other match. I also think >>that a program can be used to improve itself. It is also not at all unlikely >>that programs can think asymetrically. In other words, the algorithms change >>slightly for defense verses offense. Which brings up an interesting question... >>Should they? > > >A 20 game match tells you very little about which program is better. >If I played a 20 game match with ANY program against itself and just >label one program A and the other B, then it is very unlikely that >I will get an exactly even result. In fact, you can do your same >experiment with coin flips. Flip a coin 20 times and count heads >and tails and you will see that only occasionally will you get a >50 50 results. The longer the match, the less likely it is you will >get an exactly 50/50 results. > >It turns out that you can even get fairly lopsided results with >short matches. If you played your experiment on a 4 game match >for instance, you have a 1/8 chance of getting a 4-0 (or 0-4) score! There is a chance for a draw so the probability is not 1/8 > >If you get a book on probability and statistics you will see that >what you are describing is not strange at all, in fact it is to be >expected! Your 6.5 - 3.5 rebel score is very normal for such a >short match. That is why we must run at least a 100 games before >attaching much signficance to a result. That is also why I don't >take the Deep Blue victory very seriously, nor would I have taken >a Kasparov victory very seriously either. And that is also why >many people are asking for more rounds to be played in computer >chess events like the world championship. > >- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.