Author: Graham Banks
Date: 03:23:28 10/18/05
Go up one level in this thread
On October 18, 2005 at 06:10:05, George Tsavdaris wrote: >On October 18, 2005 at 05:59:08, Graham Banks wrote: >>>>No opening books >>>>Standings after Round 32 >>>> >>>>20.5 - D1 Meandros >>>>20.0 - WoDra >........... >........... >>>>12.5 - Solomon >>>>12.0 - Default >>>>12.0 - Cobra >>>>11.0 - Vegeta 2d >>> >>>No proof. >>> >>>number of games is not enough. >>> >>>The same program can score 12/32 in one tournament and 20/32 in another >>>tournament even without changing the time control. >>> >>Not under these conditions if you look - "no books" > > Yes, but how do you know that the results will be the same when opening books >will be set to ON? Perhaps CMDefault would be stronger from D1 Meandros for >example, when it plays with book ON....... > So you should provide a tournament with book ON and with a much higher number >of games, to be able to tell this a "proof". > But even then it would be no proof since you will have to include and non-CM >engines. And the number of these engines should be big. For example 10-20 non-CM >engines have to participate...... > >> >> >>>It is also no proof when you test only against chessmaster personalities because >>>it is possible that the default personality is worse at longer time control >>>against other CM personalities but it is not the case when you test against >>>other opponents like Fruit,Fritz,Ktulu. >> >>A good point Uri, but in my experience in testing both CM9000 and CM10th >>Edition, the majority of these settings also outperform the default settings at >>longer time controls. >>Ray can provide proof of this with both default and all settings he's tested >>having played 320 games against other programs. >>Not too much point arguing CM10th testing with us as we've been doing it a long >>time under many different scenarios. > >We will be close to call something a "proof" only when: >There is a tournament that each engine will play at least 300 or more games, it >will play with book=ON (because that is the way engines will play decent Chess >and not doubtful openings) and it will have at least 15-20 top, semi-top engines >different from Chessmaster. Just for you George! 40 games per setting against each of Shredder 9, Fruit 2.1, Aristarch 4.50, Fritz 8 Bilbao, Gandalf 6.0, Ruffian 2.1.0, Junior 9 and Hiarcs 9 That's 320 games per setting. Time control 40 moves in 40 minutes on dual Athlons, ponder on, own books, 3-4-5 men EGTB 159.0 - Cell 155.0 - Behemoth 154.5 - Milan 2.3 153.0 - Berean 5.53 150.0 - Schumacher 148.0 - Emperor 147.5 - Undertaker 3 147.5 - D1 Meandros 145.5 - Yoda 2.7 145.0 - Myrddin 143.0 - Beast 139.5 - Cobra 134.0 - Default Any other questions? Regards, Graham.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.