Author: Don Dailey
Date: 20:20:40 10/24/98
Go up one level in this thread
On October 22, 1998 at 23:49:12, blass uri wrote: > >On October 22, 1998 at 21:07:00, Don Dailey wrote: > >>On October 22, 1998 at 20:41:48, Dann Corbit wrote: >> >>>On October 22, 1998 at 17:14:19, Howard Exner wrote: >>>>Has anyone tried this ... Running a 20 game blitz match with the same >>>>program but labeling the one program A and the other B. So the first >>>>game would for example have Crafty A vs. Crafty B. Then the match would >>>>continue with Crafty B vs. Crafty A and so on. >>>> >>>>What would be the results after a 20 game match? What if many 20 >>>>game matches were run like this? Could this tell us something about >>>>the predictability of what 20 game match scores tell us? Since the >>>>programs competing are dead equal one would suspect a 10 - 10 result. >>>>I'm curious what might be the range of the match results. >>>> >>>>I tried a 10 game match like this a while ago using Rebel 8. >>>>The results of Rebel A vs Rebel B was 6.5-3.5. >>>> >>>>If someone has data on this could they post it. >>>I think that shows, without a doubt that rebel A is much stronger than rebel B >>>;-) [See my other rantings elsewhere]. >>> >>>I think this experiment is equally interesting to any other match. I also think >>>that a program can be used to improve itself. It is also not at all unlikely >>>that programs can think asymetrically. In other words, the algorithms change >>>slightly for defense verses offense. Which brings up an interesting question... >>>Should they? >> >> >>A 20 game match tells you very little about which program is better. >>If I played a 20 game match with ANY program against itself and just >>label one program A and the other B, then it is very unlikely that >>I will get an exactly even result. In fact, you can do your same >>experiment with coin flips. Flip a coin 20 times and count heads >>and tails and you will see that only occasionally will you get a >>50 50 results. The longer the match, the less likely it is you will >>get an exactly 50/50 results. >> >>It turns out that you can even get fairly lopsided results with >>short matches. If you played your experiment on a 4 game match >>for instance, you have a 1/8 chance of getting a 4-0 (or 0-4) score! > >There is a chance for a draw so the probability is not 1/8 Yes, you are right. I used the simplifying assumption and approximation that only wins and losses can happen. But my main point of course (which is grade school stuff) is that the more games you play, the LEAST likely you will end up with an exactly even score, even if you are exactly matched. - Don >> >>If you get a book on probability and statistics you will see that >>what you are describing is not strange at all, in fact it is to be >>expected! Your 6.5 - 3.5 rebel score is very normal for such a >>short match. That is why we must run at least a 100 games before >>attaching much signficance to a result. That is also why I don't >>take the Deep Blue victory very seriously, nor would I have taken >>a Kasparov victory very seriously either. And that is also why >>many people are asking for more rounds to be played in computer >>chess events like the world championship. >> >>- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.