Author: Robert Hyatt
Date: 22:13:59 01/23/99
Go up one level in this thread
On January 23, 1999 at 16:44:36, Don Dailey wrote: >On January 23, 1999 at 14:12:33, Frank Quisinsky wrote: > >>Hello, >> >>Hans is attacked to injustice! >> >>They are themselves united three programmers: -)) >>(Bruce Morland, Robert Hyatt and Don Dailey). > >I'm flattered to be considered in the same company as these >two gentlemen but I don't work with either of them on anything. > > >>The three musketeers (sorry) against Hans, interestingly! >> >>Does each new program now be attacked here? >>It could be Crafty! >> >>Maybe, it was also a mistake to give the Source code of Crafty freely. >> >>A mouse is attacked by three elephants, why ?? >> >>I like to defend the mouse! >> >> >>Don and Robert, >> >>here has developed so good programs, would also like others that. >>And the help of Robert (Source code) for other programmers >>is great however. Therefore why now attacks Hans, he was only honest. >> >>A fan of Crafty and Bionic Impakt! I didn't attack a single person. I pointed out that what was done was not exactly (a) fair to the other competitors since these guys 'borrowed' a pretty good search and a parallel search at that, with _no_ development of their own in that area; (b) Crafty is about 50K lines of code. How many did they change? 1,000? ie 2%? And that makes a _new_ program? I don't think so. > > >I reserve judgment on what happened at the tournament. I also propose >an experiment and I also have a question for Bob Hyatt. I don't doubt >Bob's results at all, I just don't really know anything about how he >ran his test and want to know what he considered a match, how many >games he looked at and how many moves in these games and actual >match percentage. The actual number of matches don't mean enough >for me to make a judgement. I don't know if 90% is reasonable to >expect or ridiculously high. Mabye most programs will match most >moves in a game with forced moves. I couldn't guess exactly how long to let crafty (my 16.1 version) search to 'match' them. They were running on a dual overclocked to something over 500mhz. Vincent Diepeveen sent me the games from the first weekend of play. I picked 3 as that was all the time I had. I had crafty search each (on my quad P6/200 which I figured was slower than the machine they used by a significant amount.) I had crafty search for 10 minutes per move, and if it chose the same move anywhere between 3 and 8 minutes as they did, I called this a match. I got all but one move in those three games. Not a 'perfect' matching scheme, but with the parallel search, it is non-trivial even on identical hardware. > >Bob notices that Bionics moves closely match some version of Crafty's >moves except in 2 cases. But the question is: Do they match as some >specified time control on some particular piece of hardware? Or is >it the case that at some point in the evaluation, Crafty would play >a move Bionic actually played? I also would like to know how many >games you actually tested this against. A single game with lots of >forcing moves might leave some room for doubt, probably most programs >would match a high percentage. I can see a possibility that if I >did a 10 minute search with Cilkchess I might match a lot of moves >that another computer might make, especially if I could match them against >ANY move Cilkchess might play during the whole search period. 3 games sent to me by Vincent from the tournament. But the most troubling thing was that after quite a bit of protesting, the next weekend's games produced results wildly different from the real 'crafty'. Bionic also did much worse that weekend, dropping out of first place. It was all 'odd' to say the least. > >Why don't several of us try this same experiment with our own chess >programs? The question I want to answer, is what is a reasonable >number of moves to match against Bionic? Maybe if one or two of >us can get a very high move matching percentage this will exonerate >Bionic? > >If anyone is interested, pick a bionic game at random from the 1st >week of play in the Dutch98 championship and report your results >here. The 1st week is the first 6 rounds. You can get the games >from the Dutch site. My site has a link to the dutch site >somewhere: "supertech.lcs.mit.edu/~dutch98" > >I suggest we start at move 20 to bypass the book and that we pick >games at random in case a particularly tactical and forcing game >needs to be seen, where a high match is likely. > >Can you give us more details Bob? I would like to either >exonerate Bionic, show that there is room for doubt, or convince >myself that they were indeed using Crafty just for my own edification >before I form a personal opinion. > >- Don the above is all I have to go on. Vincent also has crafty and he found the same thing for the first weekend, but not the second. My only gripe here is that someone took a good parallel search and used it against others. Writing one is one thing. Taking one verbatim is something else entirely. I didn't write Crafty for others to use in computer chess tournaments. I consider that _my_ earned right. I made it available so that the things I do (and don't do) can be evaluated/tried by others. Never was an intent to say "take this program, change a few lines, and then compete at a WCCC/WMCCC/ACM/etc tournament. That's always been against the rules anyway.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.