Author: pavel
Date: 10:06:51 12/18/01
Go up one level in this thread
On December 18, 2001 at 12:57:09, José Carlos wrote: >On December 18, 2001 at 12:38:15, pavel wrote: > >>> First, I'm not talking of "fair", just to make the right conclusions out of >>>the test. BTW, I don't believe there exists anything "fair" in life ... :( >>> >>>>I will state few ideas, pls let me know what you think >>>> >>>>1) Default Opening Book for the programs. >>> >>> That's usually a good idea to test "the program". But you can also test >>>program A + book X, and say "combination A+X performs this way:..." >>> >>>>(problem here, does crafty have a default opening book? the one thats available >>>>for download, is being used for a lot of versions, and not sure if it has been >>>>tweaked for craftys style of play. >>> >>> It's up to Bob to answer this question, I think. >>> >>>>Even though lets say we want to use that >>>>book, we cant use it under CB interface AFAIK. So we will have to use crafty >>>>under winboard in one computer and make it play against fritz7 with its own GUI >>>>on another computer.) >>> >>> I had an old version of Fritz (not sure the number, I don't use it anymore), >>>and I remember I created a void book with no moves, attached it to winboard >>>programs and they used their own books. >>> Also, you can download the pgn Bob used to compile the book and compile a >>>Fritz ctg or cbh or whatever it's called. >>> >>>>2) Ponder=on >>> >>> Again, you can use ponder off and report "crafty + ponder off performs..." but >>>not just say "crafty performs...". >>> >>>>3) 2 Identical CPUs >>>> >>>>(thats an obstacle I cant overcome, gotta buy another computer.) >>> >>> I only have one computer and test with ponder off. I wish I had more money... >>>:) >>> >>>>4) Tournament Time Control >>> >>> Again, if you state "program A performs ... at time control xxx on a zzz >>>machine" it's just fine :) >>> >>>>5) reasonable amount of game, 500. IMO thats reasonable enough. >>> >>> Yep. >>> >>> Regards, >>> >>> José C. >> >> >> >>pls let me know what you think. >> >>Lets say, I take these measures to perform a "more accurate" match between these >>2 programs, under these conditions. >> >> I let both programs play under fritz interface, >> >>with ponder=off >> >>(but the game will be on tournament time control 80'/40+60'/20+30, so even if >>crafty plays with ponder=off it wont suffer. Because I AFAI remember the problem >>with crafty ponder=off was with its time allocation.) >> >>with 2600.ctg >> >>(but I will observe all the games to see if any programs come out of book with a >>- score, if any programs comes out of book with more than lets say -.50 score, I >>wil discontinue the game.) >> >>And of course I will try to remember to quote "Under these circumstances", when >>I post the games next time :) >> >> >>regards. >>pavs > > If you quote "Under these circumstances", anything you do is fine for me. Any >test is interesting as long as we know what is being tested. > Of course, this is just my opinion. There seems to be other programmers here >with different opinions, so I might be completely wrong. Perhaps is better to >generalize always... > > José C. yes ofcourse, but my objective would be to try find as accurate "game environtment" as possible. Differant Issue, is it possible to make an opening book with lines not deep enough and not complicated enough, where most programs will evaluate its out of book positions as 0.00-ish ? Or rather doesnt have an instant attack on its playing style when out of book? IMO that would be interesting for games between several engines. We could call it a "Universal Opening Book". oh well, Lunch time. :) regards, pavs
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.