Author: pavel
Date: 09:38:15 12/18/01
Go up one level in this thread
> First, I'm not talking of "fair", just to make the right conclusions out of >the test. BTW, I don't believe there exists anything "fair" in life ... :( > >>I will state few ideas, pls let me know what you think >> >>1) Default Opening Book for the programs. > > That's usually a good idea to test "the program". But you can also test >program A + book X, and say "combination A+X performs this way:..." > >>(problem here, does crafty have a default opening book? the one thats available >>for download, is being used for a lot of versions, and not sure if it has been >>tweaked for craftys style of play. > > It's up to Bob to answer this question, I think. > >>Even though lets say we want to use that >>book, we cant use it under CB interface AFAIK. So we will have to use crafty >>under winboard in one computer and make it play against fritz7 with its own GUI >>on another computer.) > > I had an old version of Fritz (not sure the number, I don't use it anymore), >and I remember I created a void book with no moves, attached it to winboard >programs and they used their own books. > Also, you can download the pgn Bob used to compile the book and compile a >Fritz ctg or cbh or whatever it's called. > >>2) Ponder=on > > Again, you can use ponder off and report "crafty + ponder off performs..." but >not just say "crafty performs...". > >>3) 2 Identical CPUs >> >>(thats an obstacle I cant overcome, gotta buy another computer.) > > I only have one computer and test with ponder off. I wish I had more money... >:) > >>4) Tournament Time Control > > Again, if you state "program A performs ... at time control xxx on a zzz >machine" it's just fine :) > >>5) reasonable amount of game, 500. IMO thats reasonable enough. > > Yep. > > Regards, > > José C. pls let me know what you think. Lets say, I take these measures to perform a "more accurate" match between these 2 programs, under these conditions. I let both programs play under fritz interface, with ponder=off (but the game will be on tournament time control 80'/40+60'/20+30, so even if crafty plays with ponder=off it wont suffer. Because I AFAI remember the problem with crafty ponder=off was with its time allocation.) with 2600.ctg (but I will observe all the games to see if any programs come out of book with a - score, if any programs comes out of book with more than lets say -.50 score, I wil discontinue the game.) And of course I will try to remember to quote "Under these circumstances", when I post the games next time :) regards. pavs
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.