Author: Robert Hyatt
Date: 19:01:36 09/07/04
Go up one level in this thread
On September 07, 2004 at 21:57:06, David Dahlem wrote: >On September 07, 2004 at 21:15:45, Robert Hyatt wrote: > >>On September 07, 2004 at 20:07:42, David Dahlem wrote: >> >>>On September 07, 2004 at 19:37:56, Robert Hyatt wrote: >>> >>>>On September 07, 2004 at 19:05:11, David Dahlem wrote: >>>> >>>>>On September 07, 2004 at 18:56:11, Mathieu Pagé wrote: >>>>> >>>>>>On September 07, 2004 at 18:15:54, David Dahlem wrote: >>>>>> >>>>>>>On September 07, 2004 at 15:25:47, Robert Hyatt wrote: >>>>>>> >>>>>>>>On September 07, 2004 at 14:26:10, David Dahlem wrote: >>>>>>>> >>>>>>>>>On September 07, 2004 at 13:29:58, Robert Hyatt wrote: >>>>>>>>> >>>>>>>>>>On September 07, 2004 at 12:26:59, David Dahlem wrote: >>>>>>>>>> >>>>>>>>>>>On September 07, 2004 at 11:46:59, Axel Schumacher wrote: >>>>>>>>>>> >>>>>>>>>>>>On September 07, 2004 at 10:38:25, Cliff Sears wrote: >>>>>>>>>>>> >>>>>>>>>>>>>Has anyone done a tournament Crafty's to see if maybe one of the older versions >>>>>>>>>>>>>may be better than the newer versions? >>>>>>>>>>>>> >>>>>>>>>>>>>I seem to recall someone awile ago saying they thought Crafty 18.13 was one of >>>>>>>>>>>>>the best. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>Here the Crafty-Version I tested in my Tournament (after 133.000 games, Blitz): >>>>>>>>>>>> >>>>>>>>>>>>Crafty 16.1 2539 94 >>>>>>>>>>>>Crafty 17.13 2646 867 >>>>>>>>>>>>Crafty 17.14 2627 125 >>>>>>>>>>>>Crafty 18.10 2651 665 >>>>>>>>>>>>Crafty 18.12 2657 359 >>>>>>>>>>>>Crafty 18.13 2585 231 >>>>>>>>>>>>Crafty 18.14 2625 242 >>>>>>>>>>>>Crafty 18.15 2632 327 >>>>>>>>>>>>Crafty 19.00 2486 117 >>>>>>>>>>>>Crafty 19.01 2616 304 >>>>>>>>>>>>Crafty 19.03 2640 350 >>>>>>>>>>>>Crafty 19.03 Nimzovich 2427 15 >>>>>>>>>>>>Crafty 19.03 Petrosian 2662 48 >>>>>>>>>>>>Crafty 19.03 Stein 2668 198 >>>>>>>>>>>>Crafty 19.04 Fischer 2685 231 >>>>>>>>>>>>Crafty 19.04 Stein 2672 136 >>>>>>>>>>>>Crafty 19.04b Stein 2615 47 >>>>>>>>>>>>Crafty 19.05 2634 80 >>>>>>>>>>>>Crafty 19.06 2646 110 >>>>>>>>>>>>Crafty 19.07 2628 92 >>>>>>>>>>>>Crafty 19.07 SE 2610 57 >>>>>>>>>>>>Crafty 19.08 SE 2630 89 >>>>>>>>>>>>Crafty 19.09 2652 143 >>>>>>>>>>>>Crafty 19.10 CCT6 2663 126 >>>>>>>>>>>>Crafty 19.11 2643 105 >>>>>>>>>>>>Crafty 19.12 2627 302 >>>>>>>>>>>>Crafty 19.13 2632 196 >>>>>>>>>>>>Crafty 19.14 2616 259 >>>>>>>>>>>>Crafty 19.14 nonsmp 2586 108 >>>>>>>>>>>>Crafty 19.15 2672 373 >>>>>>>>>>>>Crafty 19.17 2621 70 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>Cheers >>>>>>>>>>>>Axel >>>>>>>>>>> >>>>>>>>>>>Hi Axel >>>>>>>>>>> >>>>>>>>>>>Which build of Crafty 19.17 did you use in this test? I am currently testing two >>>>>>>>>>>builds by Peter Skinner, an all-processor build, and a build optimized for a P3. >>>>>>>>>>>Both versions surely seem to be doing much worse than previous versions. :-( >>>>>>>>>>> >>>>>>>>>>>Regards >>>>>>>>>>>Dave >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>You need to be sure and clear position learning files before a match, and be >>>>>>>>>>sure that the two versions use different book.bin files or learning will >>>>>>>>>>definitely be broken. >>>>>>>>> >>>>>>>>>Thanks. But i am not testing one Crafty version against another Crafty version. >>>>>>>>>I am testing against different engines. And i don't use the Crafty book. I use a >>>>>>>>>custom pgn book with the Arena pgnbook option, so both engines play both sides >>>>>>>>>of each fairly equal line. >>>>>>>>> >>>>>>>>>Dave >>>>>>>> >>>>>>>> >>>>>>>>Different flaw then. Crafty has learning code. You are playing without using >>>>>>>>it.... What difference would the book make since if a commercial program plays >>>>>>>>against crafty in a real tournament, crafty will _not_ be using some half-baked >>>>>>>>book that might not fit its playing style very well? >>>>>>>> >>>>>>>>However, I only care about "best" vs "best" myself, others might be interested >>>>>>>>in some other sort of comparison, but I will never play in a WCCC event without >>>>>>>>having a reasonable book, so any other kind of match won't give much useful >>>>>>>>information IMHO... >>>>>>> >>>>>>>Well, it's just my personal opinion, but if both engines are using the same >>>>>>>book, in fact, play both sides of each line, then it's a fair match. :-) >>>>>> >>>>>>In Fact it is not, since Crafty has been handicaped. In my opinion, if you realy >>>>>>want to compare two engines you have to let them compete with all their >>>>>>functionalities enabled (Or anything the engine creator think is the best >>>>>>combination). >>>>>> >>>>>>In this case it is clear that Robert think that Crafty should use it's own book, >>>>>>since anything else would just be "Crafty without optimal opening book". >>>>>> >>>>>>I don't really understand why people think it is not fair to let the enginnes >>>>>>use any techniques they can in order to win. I have seen post here of people >>>>>>thinking that a fair match would have to be played without openings books, >>>>>>without endgames database. I even see people arguing that the memory print of >>>>>>the engines should be limited to some undreds Kb. >>>>>> >>>>>>Hey ! We are programming "Automated chess players", not only AlphaBeta searcher, >>>>>>so anything not involving human intervention during the game should be allowed >>>>>>(Humman intervention between the games of a match or a tournements are >>>>>>questionable). >>>>>> >>>>>>Mathieu Pagé >>>>> >>>>>Hi Mathieu >>>>> >>>>>I have to respectfully disagree with everything you say. And i don't see how >>>>>anyone can call using tablebases in a match as "playing", since it is only >>>>>looking up moves in a list, not playing or even thinking about the moves! >>>>> >>>>>Of course, you're entitled to your opinion, and me to mine, it's my time. And my >>>>>method of testing has proven itself over a long period of time. :-) >>>>> >>>>>Regards >>>>>Dave >>>> >>>> >>>>So? :) >>>> >>>>15th century naval explorers proved the world was flat. 18th century people >>>>proved that we could never travel to the moon because they could not fire a >>>>projectile fast enough to reach there. The list goes on and on about things >>>>that have been "proven". But if the experimental setup itself is flawed, about >>>>all that is proven is that you can prove anything you want if you overlook >>>>enough important details... >>> >>>Testing the "thinking" strength of engines is flawed? >> >>When is the last time _you_ played someone a serious game by starting off in >>some oddball position from an opening you _never_ play? >> >>About the same time that I did that I'll bet... >> >>>Why not just have human >>>created opening books that will reach the endgames, and then have tablebases >>>finish the games without any requirement for the engines to think at all!! >>> >> >> >>Math makes that impossible. But it doesn't make it impossible to at least have >>the opening book leave the program in a position that it understands reasonably >>well. IE if the engine is better in endgames, then openings that lead to quick >>endgames are better. If the program is better at attacking, then openings that >>lead to attack possibilities are better. Etc. Openings _can_ favor one engine >>over another. >> >>I used to use this strategy regularly to beat a player that was at least one >>USCF class above me. But he was stubborn enough that even though he was an >>excellent endgame player, he would let me take us into wild tactical games where >>I often out-calculated him and won. >> >>I _clearly_ wasn't better than he was, unless I could sucker him into something >>he really didn't like, due to "macho" issues he had. :) >> >>If you don't see the "problem" I doubt there is anything I can say to make it >>clearer. But there _is_ a problem with garbage books. > >I'm not using garbage books with oddball positions. Give me credit for being >smarter than that. I'm using standard opening lines from eco.pgn, with all >"oddball" unequal lines removed. > >Regards >Dave Please pay close attention. 1. Crafty never plays any Najdorf openings from _either_ side. 2. Crafty never plays castle opposite openings. 3. etc. "oddball" -> openings that Crafty will _not_ play under normal circumstances. For example, did your book have crafty play the Sicilian as black? It will _never_ do that in real games. Did it have crafty play 1. e4 as white? It will _never_ do that in real games. The list goes on and on. And that is my _point_. I would personally _never_ play serious games using your favorite opening books, any more than you would do the same with _my_ opening choices. Why, then, should a program that was written/tuned by me, for the specific kinds of openings I think it wil play, then have to play _other_ openings, perhaps opening systems that I don't play and therefore I don't try to handle in opening-specific code such as EvaluateDevelopment()??? I think this is an open and shut argument about producing results that are not very meaningful...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.