Author: Volker Böhm
Date: 08:02:03 03/30/05
Go up one level in this thread
On March 30, 2005 at 08:50:45, Heinz van Kempen wrote: >Hi all :) , > >we updated our CEGT websites again and now have 859 games for most engines. The >rating list is here: > >http://www.husvankempen.de/nunn/cegt.html > >The list itself is not astonishing, but when one compares total results from >CEGT 1 with 510 games for each engine with those of CEGT 2 with 349 games for >each engine, testers will have to confess that it is practically impossible to >measure the strength of engines with less than 1000 games and we should stop to >draw conclusions from shorter tournaments. Just compare Junior and Fritz or >Gandalf in CEGT 1 and 2. > >This is independent of time control, although I hoped it would be less extreme >with our longer time control. It is also independent of using Nunn, Noomen or >general books, as we now used almost all Nunn positions to cover the most >important openings and Charles with general books also has the same differences. > >We will push the CEGT tournaments to more than 1000 games for each engine to >reduce the error bars. > >http://www.chessfighters.de/cegt/ >http://www.husvankempen.de/nunn/ > >Please also read about AEGT starting these days. > >Best Regards >Heinz Hi Heinz, for Spike we play about 1000 Games to decide for a change. The 1000 Game are played versus 10 different opponents with 50 standard openings (thus playing without book). We´ve seen that time-control, opponents does matter. Even then 10-20 elo can be hazardous. It is especially hard to compare engines if they only plays against each other with opening books switched on. I suggest to look at games to see if two engine often played the same game until a position that is a clear loss/win for one side (The book builder of spike is well suited to do that :-). Which value has a statistic of 1000 games with only 100 different games? With a larger amount of opponents you can ignore this effect. Greetings Volker
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.