Author: Don Dailey
Date: 11:42:47 01/27/99
Go up one level in this thread
On January 26, 1999 at 15:21:34, Bruce Moreland wrote: > >On January 26, 1999 at 13:49:29, Don Dailey wrote: > >>Hi Dann, >> >>I'm not trying to prove guilt, I'm trying to prove innocence. I don't >>think this test can prove guilt but it can easily raise enough doubt >>that we should back off. >> >>If you are right and all or even one of the programs give a high match, >>then there is plenty of room for doubt and we give them the benefit of >>the doubt which I think is fair. If however, Crafty matches >>significantly higher percentage, then we have something to talk about. >>I won't be claiming guilt in this case, just that there is still >>room to talk. >> >>I have already discovered that running 60 seconds on Cilkchess will >>guarantee a much lower match rate, so I am a little concerned that >>people will run this very short 60 second test and try to draw unfair >>conclusions from it. >> >>We will have enough data to come to some conclusion, even if the >>conclusion is that we are not sure! We also have log files from >>them and an executable. So we should be able to come to some >>intelligent conclusion. If nothing else, I want to know for my >>own edification. > >I do not know what the match rates will be for programs in general, or if there >will be any identifiable patterns in the match rate. > >I don't know what the match rate for those different versions of Crafty will be, >but I'm very interested in finding out. > >We have a contention, from Bob, that the match rate was 100% for three of those >games, given sufficient time and some loose interpretation of a what a match is. > >I would like to see if the match rate is high for Crafty even at a minute per >move. > >If it is extremely high, it does point to some conclusions, and may provide some >indication of what additional tests are necessary. > >If it is not extremely high, or if there is no difference between those three >games and the other 8 for Crafty, it may put some pressure on Bob to repeat his >original experiment or retract what amounts to some very serious charges. > >Mine had an 88% match rate in one of the games (15/17). That's interesting, and >if enough other people do this experiment, perhaps we'll see this as a trend in >that game, or in other games, or whatever. > >This experiment isn't perfect, but it will only take 14 hours of computer time, >and I'm willing to do the work to compile the data into readable form. > >It may very well be possible that there is nothing to conclude. It also may be >that you will wish to run your 10-minute test as well. I would encourage this, >but please run mine first. > >If your 10-minute results are interesting when compared with the 1-minute >results, maybe you'll get people to do the 10-minute test as well. > >I'm not trying to kill anybody here. I just want some data, and I think that >this is a good opportunity to get some. We have people arguing various >contentions, you'd think that people would want to say something concrete. > >bruce Yes, let's just get the data and then talk about it. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.