Author: Don Dailey
Date: 18:07:00 10/22/98
Go up one level in this thread
On October 22, 1998 at 20:41:48, Dann Corbit wrote: >On October 22, 1998 at 17:14:19, Howard Exner wrote: >>Has anyone tried this ... Running a 20 game blitz match with the same >>program but labeling the one program A and the other B. So the first >>game would for example have Crafty A vs. Crafty B. Then the match would >>continue with Crafty B vs. Crafty A and so on. >> >>What would be the results after a 20 game match? What if many 20 >>game matches were run like this? Could this tell us something about >>the predictability of what 20 game match scores tell us? Since the >>programs competing are dead equal one would suspect a 10 - 10 result. >>I'm curious what might be the range of the match results. >> >>I tried a 10 game match like this a while ago using Rebel 8. >>The results of Rebel A vs Rebel B was 6.5-3.5. >> >>If someone has data on this could they post it. >I think that shows, without a doubt that rebel A is much stronger than rebel B >;-) [See my other rantings elsewhere]. > >I think this experiment is equally interesting to any other match. I also think >that a program can be used to improve itself. It is also not at all unlikely >that programs can think asymetrically. In other words, the algorithms change >slightly for defense verses offense. Which brings up an interesting question... >Should they? A 20 game match tells you very little about which program is better. If I played a 20 game match with ANY program against itself and just label one program A and the other B, then it is very unlikely that I will get an exactly even result. In fact, you can do your same experiment with coin flips. Flip a coin 20 times and count heads and tails and you will see that only occasionally will you get a 50 50 results. The longer the match, the less likely it is you will get an exactly 50/50 results. It turns out that you can even get fairly lopsided results with short matches. If you played your experiment on a 4 game match for instance, you have a 1/8 chance of getting a 4-0 (or 0-4) score! If you get a book on probability and statistics you will see that what you are describing is not strange at all, in fact it is to be expected! Your 6.5 - 3.5 rebel score is very normal for such a short match. That is why we must run at least a 100 games before attaching much signficance to a result. That is also why I don't take the Deep Blue victory very seriously, nor would I have taken a Kasparov victory very seriously either. And that is also why many people are asking for more rounds to be played in computer chess events like the world championship. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.