Author: Uri Blass
Date: 05:19:57 01/26/06
Go up one level in this thread
On January 26, 2006 at 07:53:39, Joseph Ciarrochi wrote: > >> >>I think that it is a bad comparison because the CEGT use different positions and >>not the noomen test. >> >>The only good way to compare is if the same computer and the same positions >>are used for the first version and the last version. >> >>Uri > >yes, i am planning to run beta 1 on the same computer against the same >opponents. I agree this is the best control > >However, your use of terms like "bad comparison" and "the only good way" >suggests that there is no generality across computers and opening sets. I am >sure despite your language you don't mean anythything quite this extreme? of course there is correlation but when the difference is small it is better to avoid factors that can change the result like different computers. James walker tests are relatively better because they are done in the same computer. In theory there may be also disadvantages of his tests because of not having the same positions but probably these disadvantages has little influence. This >sort of language would imply that you can't really ever say one engine is >stronger than another, since it would be "entirely" conditional on opening set >and computer. If you see that one engine is better on computers of many testers or you see that it is significantly better(more than 100 elo difference after enough games and not because of learning) then you can practically say that one engine is better. > >Actually, my experience has been that the results are quite consistent, if you >hold time controls constant. I agree that there is a good correlation but when there is a small difference(and 40/4 minutes are less sensitive to time management improvements relative to 5 minutes per game) then I think that it is better to say nothing based on comparison between different computers and different condition of positions. My opinion is that it is better even to use books in both cases like James walker does and not to compare between Noomen match and match that is based on different positions(my opinion is that the best comparison is without books from the same positions but of course people have the right to use books). The reason that I am interested in comparison without book is that I use engines mainly for analysis of my correspondence games and better book is simply irrelevant for my decisions which engine to use. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.