Author: Wayne Lowrance
Date: 17:17:11 01/26/06
Go up one level in this thread
On January 26, 2006 at 08:19:57, Uri Blass wrote: >On January 26, 2006 at 07:53:39, Joseph Ciarrochi wrote: > >> >>> >>>I think that it is a bad comparison because the CEGT use different positions and >>>not the noomen test. >>> >>>The only good way to compare is if the same computer and the same positions >>>are used for the first version and the last version. >>> >>>Uri >> >>yes, i am planning to run beta 1 on the same computer against the same >>opponents. I agree this is the best control >> >>However, your use of terms like "bad comparison" and "the only good way" >>suggests that there is no generality across computers and opening sets. I am >>sure despite your language you don't mean anythything quite this extreme? > >of course there is correlation but when the difference is small it is better to >avoid factors that can change the result like different computers. > >James walker tests are relatively better because they are done in the same >computer. > >In theory there may be also disadvantages of his tests because of not having the >same positions but probably these disadvantages has little influence. > > > This >>sort of language would imply that you can't really ever say one engine is >>stronger than another, since it would be "entirely" conditional on opening set >>and computer. > >If you see that one engine is better on computers of many testers or you see >that it is significantly better(more than 100 elo difference after enough games >and not because of learning) then you can practically say that one engine is >better. > >> >>Actually, my experience has been that the results are quite consistent, if you >>hold time controls constant. > >I agree that there is a good correlation but when there is a small >difference(and 40/4 minutes are less sensitive to time management improvements >relative to 5 minutes per game) then I think that it is better to say nothing >based on comparison between different computers and different condition of >positions. > >My opinion is that it is better even to use books in both cases like James >walker does and not to compare between Noomen match and match that is based on >different positions(my opinion is that the best comparison is without books from >the same positions but of course people have the right to use books). > >The reason that I am interested in comparison without book is that I use engines >mainly for analysis of my correspondence games and better book is simply >irrelevant for my decisions which engine to use. > have you ever visited the CC location ?http://www.pacific-mall.com/ Wayne
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.