Author: José Carlos
Date: 10:25:22 12/18/01
Go up one level in this thread
On December 18, 2001 at 13:06:51, pavel wrote: >On December 18, 2001 at 12:57:09, José Carlos wrote: > >>On December 18, 2001 at 12:38:15, pavel wrote: >> >>>> First, I'm not talking of "fair", just to make the right conclusions out of >>>>the test. BTW, I don't believe there exists anything "fair" in life ... :( >>>> >>>>>I will state few ideas, pls let me know what you think >>>>> >>>>>1) Default Opening Book for the programs. >>>> >>>> That's usually a good idea to test "the program". But you can also test >>>>program A + book X, and say "combination A+X performs this way:..." >>>> >>>>>(problem here, does crafty have a default opening book? the one thats available >>>>>for download, is being used for a lot of versions, and not sure if it has been >>>>>tweaked for craftys style of play. >>>> >>>> It's up to Bob to answer this question, I think. >>>> >>>>>Even though lets say we want to use that >>>>>book, we cant use it under CB interface AFAIK. So we will have to use crafty >>>>>under winboard in one computer and make it play against fritz7 with its own GUI >>>>>on another computer.) >>>> >>>> I had an old version of Fritz (not sure the number, I don't use it anymore), >>>>and I remember I created a void book with no moves, attached it to winboard >>>>programs and they used their own books. >>>> Also, you can download the pgn Bob used to compile the book and compile a >>>>Fritz ctg or cbh or whatever it's called. >>>> >>>>>2) Ponder=on >>>> >>>> Again, you can use ponder off and report "crafty + ponder off performs..." but >>>>not just say "crafty performs...". >>>> >>>>>3) 2 Identical CPUs >>>>> >>>>>(thats an obstacle I cant overcome, gotta buy another computer.) >>>> >>>> I only have one computer and test with ponder off. I wish I had more money... >>>>:) >>>> >>>>>4) Tournament Time Control >>>> >>>> Again, if you state "program A performs ... at time control xxx on a zzz >>>>machine" it's just fine :) >>>> >>>>>5) reasonable amount of game, 500. IMO thats reasonable enough. >>>> >>>> Yep. >>>> >>>> Regards, >>>> >>>> José C. >>> >>> >>> >>>pls let me know what you think. >>> >>>Lets say, I take these measures to perform a "more accurate" match between these >>>2 programs, under these conditions. >>> >>> I let both programs play under fritz interface, >>> >>>with ponder=off >>> >>>(but the game will be on tournament time control 80'/40+60'/20+30, so even if >>>crafty plays with ponder=off it wont suffer. Because I AFAI remember the problem >>>with crafty ponder=off was with its time allocation.) >>> >>>with 2600.ctg >>> >>>(but I will observe all the games to see if any programs come out of book with a >>>- score, if any programs comes out of book with more than lets say -.50 score, I >>>wil discontinue the game.) >>> >>>And of course I will try to remember to quote "Under these circumstances", when >>>I post the games next time :) >>> >>> >>>regards. >>>pavs >> >> If you quote "Under these circumstances", anything you do is fine for me. Any >>test is interesting as long as we know what is being tested. >> Of course, this is just my opinion. There seems to be other programmers here >>with different opinions, so I might be completely wrong. Perhaps is better to >>generalize always... >> >> José C. > > >yes ofcourse, but my objective would be to try find as accurate "game >environtment" as possible. > >Differant Issue, is it possible to make an opening book with lines not deep >enough and not complicated enough, where most programs will evaluate its out of >book positions as 0.00-ish ? Or rather doesnt have an instant attack on its >playing style when out of book? > >IMO that would be interesting for games between several engines. We could call >it a "Universal Opening Book". > > > >oh well, Lunch time. :) > >regards, > >pavs Every test is interesting, IMO. What about testing 'book strength' by running games between two different books with the same engine? :) Just kidding... José C.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.