Author: Robert Hyatt
Date: 14:21:12 01/24/99
Go up one level in this thread
On January 24, 1999 at 13:49:04, Don Dailey wrote: >>I didn't attack a single person. I pointed out that what was done was not >>exactly (a) fair to the other competitors since these guys 'borrowed' a pretty >>good search and a parallel search at that, with _no_ development of their own >>in that area; (b) Crafty is about 50K lines of code. How many did they >>change? 1,000? ie 2%? And that makes a _new_ program? I don't think so. > >I have thought and rethought this one and I have to say that I agree >with this sentiment now. After all, these things, although lots of >fun, are competitions. Who wants to come to a tournament and discover >that you may have written most of the program that might beat your >entry and you did all the work? > >Plus, a big part of these tournaments for me is to chat with the other >programmers and exchange ideas. I would hate to go to a tournament >and sit across from several contestants who have nothing to contribute >excepth, "oh, just take public domain program x and do such and such to it." > > > >>Vincent Diepeveen sent me the games from the first weekend of play. I picked >>3 as that was all the time I had. I had crafty search each (on my quad P6/200 >>which I figured was slower than the machine they used by a significant amount.) >>I had crafty search for 10 minutes per move, and if it chose the same move >>anywhere between 3 and 8 minutes as they did, I called this a match. I got >>all but one move in those three games. Not a 'perfect' matching scheme, but >>with the parallel search, it is non-trivial even on identical hardware. > >Ok, good this seems like a pretty reasonable way to test. There is >some issue about the book, but I'll do the same test with Cilkchess >just to convince myself one way or the other. I'll take a wild guess >and predict a cilkchess match about 70% using your matching rules. >I would point out that any matching rules by this are by neccessity >fairly liberal (no other reasonable way to do it though.) I think >this kind of test is better for proving innocence than it is proving >guilt because there could be many matches by accident. > >I realize this isn't the whole issue in your mind. I'm more >interested in the issue of whether they actually used an >unmodified Crafty since they at least were up front about >using the core program. I agree that this should not be >allowed at all in the future. > > >>the above is all I have to go on. Vincent also has crafty and he found the >>same thing for the first weekend, but not the second. > >>My only gripe here is that someone took a good parallel search and used it >>against others. Writing one is one thing. Taking one verbatim is something >>else entirely. I didn't write Crafty for others to use in computer chess >>tournaments. I consider that _my_ earned right. I made it available so that >>the things I do (and don't do) can be evaluated/tried by others. Never was an >>intent to say "take this program, change a few lines, and then compete at a >>WCCC/WMCCC/ACM/etc tournament. That's always been against the rules anyway. > >I completely agree with you Bob, but I didn't get the impression they >were hiding anything. They didn't even seem interested in getting the >credit (I talked to Hans at the tournament.) The rule should be that >in the future, if anyone does this, it should be considered a joint >experiment with joint authorship and that the principle author should >be involved (at a minimum by giving permission and being willing to >have his name attached to the project.) > >From what Hans told me, I was under the impression you knew what was >going on and had given him some advice and tips. Was this done >completely without your knowledge? I also don't understand the >first week using a pure Crafty version if this is what happened. >Hans didn't appear to be interested in this version, only his own. >I would really like to know what REALLY happened in some detail >because none of it seems completely plausible. The only possible >middle ground is that he used Crafty completely by accident, which >I don't really buy (and he never claimed this anyway.) first let me qualify my answer: I average 50-100 emails _per day_, with 75% being about crafty, 25% about the stuff I do with Linux. It is very possible someone asked me how to do something and I told them. But I do read my email, and if someone had mentioned taking the program and modifiying it and then using it in a computer chess event, I would have picked up on that. So I _may_ have talked with them via email, I save every message I send, but that averages about 2500 per month, so I don't want to go back and sort thru them to see what is there... To the best of my knowledge, then, I haven't been asked about using crafty in such a manner. But keep the disclaimer above in mind... > >But can we give him the benefit of the doubt for just a little >while and see if other chess programs can get a high match rate? > >An interesting experiment you could also do is try different >versions of Crafty (perhaps you have already tried this.) Does >the match rate change a lot with different versions (with different >evaluation functions?) If they do, this is an argument against >the possiblity that you match a lot because he started with your >search, your extensions and the very same evaluation terms. He >told me that he added a pre-processor (he didn't call it this) >and that this was a significant component. He also claimed >that this made a really large difference in playing strength. > yes. IN fact vincent was sure it was the _current_ version at the time, because he had played dozens of long games against Crafty getting ready for this event... Vincent's a very good chess player, and he pretty well knows what kinds of things Crafty will and won't do in a given situation. I tried a late 15.x (maybe 15.15 or so) and didn't get nearly so good a match as I did with 16.1 (I think .1, maybe .0 at the time...) Of course, this could still be circumstantial, but my impression was that the moves played by Bionic the first weekend looked similar to what Crafty would likely play given the same hardware/time. On the second weekend this wasn't true, as Bionic made several moves crafty classified as outright blunders. No explanation for that by me... I only noted that the first game Vincent suggested I look at was really ugly... and Bionic lost that game badly... I don't remember the opponent however... > > >- Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.