Author: Robert Hyatt
Date: 18:15:45 09/07/04
Go up one level in this thread
On September 07, 2004 at 20:07:42, David Dahlem wrote: >On September 07, 2004 at 19:37:56, Robert Hyatt wrote: > >>On September 07, 2004 at 19:05:11, David Dahlem wrote: >> >>>On September 07, 2004 at 18:56:11, Mathieu Pagé wrote: >>> >>>>On September 07, 2004 at 18:15:54, David Dahlem wrote: >>>> >>>>>On September 07, 2004 at 15:25:47, Robert Hyatt wrote: >>>>> >>>>>>On September 07, 2004 at 14:26:10, David Dahlem wrote: >>>>>> >>>>>>>On September 07, 2004 at 13:29:58, Robert Hyatt wrote: >>>>>>> >>>>>>>>On September 07, 2004 at 12:26:59, David Dahlem wrote: >>>>>>>> >>>>>>>>>On September 07, 2004 at 11:46:59, Axel Schumacher wrote: >>>>>>>>> >>>>>>>>>>On September 07, 2004 at 10:38:25, Cliff Sears wrote: >>>>>>>>>> >>>>>>>>>>>Has anyone done a tournament Crafty's to see if maybe one of the older versions >>>>>>>>>>>may be better than the newer versions? >>>>>>>>>>> >>>>>>>>>>>I seem to recall someone awile ago saying they thought Crafty 18.13 was one of >>>>>>>>>>>the best. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Here the Crafty-Version I tested in my Tournament (after 133.000 games, Blitz): >>>>>>>>>> >>>>>>>>>>Crafty 16.1 2539 94 >>>>>>>>>>Crafty 17.13 2646 867 >>>>>>>>>>Crafty 17.14 2627 125 >>>>>>>>>>Crafty 18.10 2651 665 >>>>>>>>>>Crafty 18.12 2657 359 >>>>>>>>>>Crafty 18.13 2585 231 >>>>>>>>>>Crafty 18.14 2625 242 >>>>>>>>>>Crafty 18.15 2632 327 >>>>>>>>>>Crafty 19.00 2486 117 >>>>>>>>>>Crafty 19.01 2616 304 >>>>>>>>>>Crafty 19.03 2640 350 >>>>>>>>>>Crafty 19.03 Nimzovich 2427 15 >>>>>>>>>>Crafty 19.03 Petrosian 2662 48 >>>>>>>>>>Crafty 19.03 Stein 2668 198 >>>>>>>>>>Crafty 19.04 Fischer 2685 231 >>>>>>>>>>Crafty 19.04 Stein 2672 136 >>>>>>>>>>Crafty 19.04b Stein 2615 47 >>>>>>>>>>Crafty 19.05 2634 80 >>>>>>>>>>Crafty 19.06 2646 110 >>>>>>>>>>Crafty 19.07 2628 92 >>>>>>>>>>Crafty 19.07 SE 2610 57 >>>>>>>>>>Crafty 19.08 SE 2630 89 >>>>>>>>>>Crafty 19.09 2652 143 >>>>>>>>>>Crafty 19.10 CCT6 2663 126 >>>>>>>>>>Crafty 19.11 2643 105 >>>>>>>>>>Crafty 19.12 2627 302 >>>>>>>>>>Crafty 19.13 2632 196 >>>>>>>>>>Crafty 19.14 2616 259 >>>>>>>>>>Crafty 19.14 nonsmp 2586 108 >>>>>>>>>>Crafty 19.15 2672 373 >>>>>>>>>>Crafty 19.17 2621 70 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Cheers >>>>>>>>>>Axel >>>>>>>>> >>>>>>>>>Hi Axel >>>>>>>>> >>>>>>>>>Which build of Crafty 19.17 did you use in this test? I am currently testing two >>>>>>>>>builds by Peter Skinner, an all-processor build, and a build optimized for a P3. >>>>>>>>>Both versions surely seem to be doing much worse than previous versions. :-( >>>>>>>>> >>>>>>>>>Regards >>>>>>>>>Dave >>>>>>>> >>>>>>>> >>>>>>>>You need to be sure and clear position learning files before a match, and be >>>>>>>>sure that the two versions use different book.bin files or learning will >>>>>>>>definitely be broken. >>>>>>> >>>>>>>Thanks. But i am not testing one Crafty version against another Crafty version. >>>>>>>I am testing against different engines. And i don't use the Crafty book. I use a >>>>>>>custom pgn book with the Arena pgnbook option, so both engines play both sides >>>>>>>of each fairly equal line. >>>>>>> >>>>>>>Dave >>>>>> >>>>>> >>>>>>Different flaw then. Crafty has learning code. You are playing without using >>>>>>it.... What difference would the book make since if a commercial program plays >>>>>>against crafty in a real tournament, crafty will _not_ be using some half-baked >>>>>>book that might not fit its playing style very well? >>>>>> >>>>>>However, I only care about "best" vs "best" myself, others might be interested >>>>>>in some other sort of comparison, but I will never play in a WCCC event without >>>>>>having a reasonable book, so any other kind of match won't give much useful >>>>>>information IMHO... >>>>> >>>>>Well, it's just my personal opinion, but if both engines are using the same >>>>>book, in fact, play both sides of each line, then it's a fair match. :-) >>>> >>>>In Fact it is not, since Crafty has been handicaped. In my opinion, if you realy >>>>want to compare two engines you have to let them compete with all their >>>>functionalities enabled (Or anything the engine creator think is the best >>>>combination). >>>> >>>>In this case it is clear that Robert think that Crafty should use it's own book, >>>>since anything else would just be "Crafty without optimal opening book". >>>> >>>>I don't really understand why people think it is not fair to let the enginnes >>>>use any techniques they can in order to win. I have seen post here of people >>>>thinking that a fair match would have to be played without openings books, >>>>without endgames database. I even see people arguing that the memory print of >>>>the engines should be limited to some undreds Kb. >>>> >>>>Hey ! We are programming "Automated chess players", not only AlphaBeta searcher, >>>>so anything not involving human intervention during the game should be allowed >>>>(Humman intervention between the games of a match or a tournements are >>>>questionable). >>>> >>>>Mathieu Pagé >>> >>>Hi Mathieu >>> >>>I have to respectfully disagree with everything you say. And i don't see how >>>anyone can call using tablebases in a match as "playing", since it is only >>>looking up moves in a list, not playing or even thinking about the moves! >>> >>>Of course, you're entitled to your opinion, and me to mine, it's my time. And my >>>method of testing has proven itself over a long period of time. :-) >>> >>>Regards >>>Dave >> >> >>So? :) >> >>15th century naval explorers proved the world was flat. 18th century people >>proved that we could never travel to the moon because they could not fire a >>projectile fast enough to reach there. The list goes on and on about things >>that have been "proven". But if the experimental setup itself is flawed, about >>all that is proven is that you can prove anything you want if you overlook >>enough important details... > >Testing the "thinking" strength of engines is flawed? When is the last time _you_ played someone a serious game by starting off in some oddball position from an opening you _never_ play? About the same time that I did that I'll bet... >Why not just have human >created opening books that will reach the endgames, and then have tablebases >finish the games without any requirement for the engines to think at all!! > Math makes that impossible. But it doesn't make it impossible to at least have the opening book leave the program in a position that it understands reasonably well. IE if the engine is better in endgames, then openings that lead to quick endgames are better. If the program is better at attacking, then openings that lead to attack possibilities are better. Etc. Openings _can_ favor one engine over another. I used to use this strategy regularly to beat a player that was at least one USCF class above me. But he was stubborn enough that even though he was an excellent endgame player, he would let me take us into wild tactical games where I often out-calculated him and won. I _clearly_ wasn't better than he was, unless I could sucker him into something he really didn't like, due to "macho" issues he had. :) If you don't see the "problem" I doubt there is anything I can say to make it clearer. But there _is_ a problem with garbage books. >Regards >Dave
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.