Author: Don Dailey
Date: 10:49:04 01/24/99
Go up one level in this thread
>I didn't attack a single person. I pointed out that what was done was not >exactly (a) fair to the other competitors since these guys 'borrowed' a pretty >good search and a parallel search at that, with _no_ development of their own >in that area; (b) Crafty is about 50K lines of code. How many did they >change? 1,000? ie 2%? And that makes a _new_ program? I don't think so. I have thought and rethought this one and I have to say that I agree with this sentiment now. After all, these things, although lots of fun, are competitions. Who wants to come to a tournament and discover that you may have written most of the program that might beat your entry and you did all the work? Plus, a big part of these tournaments for me is to chat with the other programmers and exchange ideas. I would hate to go to a tournament and sit across from several contestants who have nothing to contribute excepth, "oh, just take public domain program x and do such and such to it." >Vincent Diepeveen sent me the games from the first weekend of play. I picked >3 as that was all the time I had. I had crafty search each (on my quad P6/200 >which I figured was slower than the machine they used by a significant amount.) >I had crafty search for 10 minutes per move, and if it chose the same move >anywhere between 3 and 8 minutes as they did, I called this a match. I got >all but one move in those three games. Not a 'perfect' matching scheme, but >with the parallel search, it is non-trivial even on identical hardware. Ok, good this seems like a pretty reasonable way to test. There is some issue about the book, but I'll do the same test with Cilkchess just to convince myself one way or the other. I'll take a wild guess and predict a cilkchess match about 70% using your matching rules. I would point out that any matching rules by this are by neccessity fairly liberal (no other reasonable way to do it though.) I think this kind of test is better for proving innocence than it is proving guilt because there could be many matches by accident. I realize this isn't the whole issue in your mind. I'm more interested in the issue of whether they actually used an unmodified Crafty since they at least were up front about using the core program. I agree that this should not be allowed at all in the future. >the above is all I have to go on. Vincent also has crafty and he found the >same thing for the first weekend, but not the second. >My only gripe here is that someone took a good parallel search and used it >against others. Writing one is one thing. Taking one verbatim is something >else entirely. I didn't write Crafty for others to use in computer chess >tournaments. I consider that _my_ earned right. I made it available so that >the things I do (and don't do) can be evaluated/tried by others. Never was an >intent to say "take this program, change a few lines, and then compete at a >WCCC/WMCCC/ACM/etc tournament. That's always been against the rules anyway. I completely agree with you Bob, but I didn't get the impression they were hiding anything. They didn't even seem interested in getting the credit (I talked to Hans at the tournament.) The rule should be that in the future, if anyone does this, it should be considered a joint experiment with joint authorship and that the principle author should be involved (at a minimum by giving permission and being willing to have his name attached to the project.) From what Hans told me, I was under the impression you knew what was going on and had given him some advice and tips. Was this done completely without your knowledge? I also don't understand the first week using a pure Crafty version if this is what happened. Hans didn't appear to be interested in this version, only his own. I would really like to know what REALLY happened in some detail because none of it seems completely plausible. The only possible middle ground is that he used Crafty completely by accident, which I don't really buy (and he never claimed this anyway.) But can we give him the benefit of the doubt for just a little while and see if other chess programs can get a high match rate? An interesting experiment you could also do is try different versions of Crafty (perhaps you have already tried this.) Does the match rate change a lot with different versions (with different evaluation functions?) If they do, this is an argument against the possiblity that you match a lot because he started with your search, your extensions and the very same evaluation terms. He told me that he added a pre-processor (he didn't call it this) and that this was a significant component. He also claimed that this made a really large difference in playing strength. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.