Author: Richard Pijl
Date: 08:57:11 12/02/02
Go up one level in this thread
On December 02, 2002 at 03:41:24, Uri Blass wrote: >On December 02, 2002 at 03:08:56, Jouni Uski wrote: > >>Isn't it always close match against older version of same program? I remember, >>that >>usually new Fritz version loses to older version, even if they are stronger >>against >>other engines. >> >>Jouni > >No Well, it could be true that differences in testing against previous versions could yield less than against other engines. It really depends on the modifications made. I noticed both could happen. > >I read the opposite. > >I read claims that the testing against previous version increase the real >difference and after new Terra beated old Terra convincingly the programmer >claimed that the difference against other programs is smaller. I've noticed this especially when knowledge in evaluation is added to fix a problem in the previous version. Then it might also get the ability to exploit that weakness itself. But the same addition my result in the opposite: If the playing style required to exploit the weakness doesn't match the abilities of the engine no progress will be measured and even reduced playing strength can be experienced, for example because the engine became slower. Conclusion: Playing against previous versions of the same engine should never be the only method for measuring improvement. And engines used for testing should have a variety of playing styles (if you test only against e.g. Sjeng, Tao and Comet you might end up with the best defending engine, not with an overal stronger one) > >I never believed in this assumption and I see no reason to assume that the >difference against other programs is smaller(the only exception is if I add >killer lines in the book against the old version). > >Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.