Author: Bruce Moreland
Date: 12:21:34 01/26/99
Go up one level in this thread
On January 26, 1999 at 13:49:29, Don Dailey wrote: >Hi Dann, > >I'm not trying to prove guilt, I'm trying to prove innocence. I don't >think this test can prove guilt but it can easily raise enough doubt >that we should back off. > >If you are right and all or even one of the programs give a high match, >then there is plenty of room for doubt and we give them the benefit of >the doubt which I think is fair. If however, Crafty matches >significantly higher percentage, then we have something to talk about. >I won't be claiming guilt in this case, just that there is still >room to talk. > >I have already discovered that running 60 seconds on Cilkchess will >guarantee a much lower match rate, so I am a little concerned that >people will run this very short 60 second test and try to draw unfair >conclusions from it. > >We will have enough data to come to some conclusion, even if the >conclusion is that we are not sure! We also have log files from >them and an executable. So we should be able to come to some >intelligent conclusion. If nothing else, I want to know for my >own edification. I do not know what the match rates will be for programs in general, or if there will be any identifiable patterns in the match rate. I don't know what the match rate for those different versions of Crafty will be, but I'm very interested in finding out. We have a contention, from Bob, that the match rate was 100% for three of those games, given sufficient time and some loose interpretation of a what a match is. I would like to see if the match rate is high for Crafty even at a minute per move. If it is extremely high, it does point to some conclusions, and may provide some indication of what additional tests are necessary. If it is not extremely high, or if there is no difference between those three games and the other 8 for Crafty, it may put some pressure on Bob to repeat his original experiment or retract what amounts to some very serious charges. Mine had an 88% match rate in one of the games (15/17). That's interesting, and if enough other people do this experiment, perhaps we'll see this as a trend in that game, or in other games, or whatever. This experiment isn't perfect, but it will only take 14 hours of computer time, and I'm willing to do the work to compile the data into readable form. It may very well be possible that there is nothing to conclude. It also may be that you will wish to run your 10-minute test as well. I would encourage this, but please run mine first. If your 10-minute results are interesting when compared with the 1-minute results, maybe you'll get people to do the 10-minute test as well. I'm not trying to kill anybody here. I just want some data, and I think that this is a good opportunity to get some. We have people arguing various contentions, you'd think that people would want to say something concrete. bruce
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.