Author: Dann Corbit
Date: 18:00:05 07/15/05
Go up one level in this thread
On July 15, 2005 at 20:38:04, Sune Fischer wrote: >On July 15, 2005 at 20:07:46, Dann Corbit wrote: [snip] >>If I have ten games with a new engine of 3000 Elo and I have 10,000 games with >>an old engine of 2300 Elo, the old engine will give me much better data by >>playing against it than against the new one. The new engine will have huge >>error bars in the confidence interval, and these must necessarily translate to >>the engine for which the calibrated engine is used as a reference. I exaggerate >>the numbers to make the meaning obvious, but the message is plain enough -- you >>get better numbers from the measuring sticks with the finest graduations on >>them. > >Here you simply end op with an ever growing list of engines you have to keep >playing against, and because they play more and more games it will be harder and >harder to take them out. How do you break this circle? There is no value to break the circle until the engine is no longer strong enough (e.g. when it becomes 300 Elo weaker than the engine that you wish to calibrate, then the data loses its punch). As long as the engine is within 100 Elo of the test engine, the more games that have been played against an engine, the better it is to test with. If we had an engine with 1 trillion games against it by carefully calibrated known opponents and it was within 100 Elo of our engine, it would be the best possible measuring line. >Take out the old engines and just play on against its opponents, there must be >many of those. The old hardware programs in the list don't have much value anymore, because most good amateur programs can clobber them. But Fritz 5.32 on a 450 MHz box will be plenty strong of an opponent for a couple years to come, probably. [snip]
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.