Author: Christophe Theron
Date: 18:14:14 12/11/99
Go up one level in this thread
On December 11, 1999 at 20:18:45, John Warfield wrote:
>On December 11, 1999 at 19:46:50, Bertil Eklund wrote:
>
>>On December 11, 1999 at 17:52:56, Tom King wrote:
>>
>>>Which of the well known test suites predicts the strength of chess programs most
>>>accurately?
>>>
>>>I ask this, because I recently made some *slight* mods. to the evaluation
>>>function in my program, Francesca. I ran the LCT-2 suite, and the results
>>>indicated that it was a wash - the modification gave me about 5 ELO points,
>>>apparently.
>>>
>>>I then ran a series of fast games against another amateur program. I realize
>>>it's important to play a large number of games, to reduce the margin of error,
>>>so I ran two matches of 65 games. The result was this:
>>>
>>>MATCH 1
>>>"Normal" Francesca scored 37% against the amateur program.
>>>
>>>MATCH 2
>>>"Modified" Francesca scored 45% against the amateur program.
>>>
>>>Quite a difference! It implies that the modification is worth over 50 ELO. I
>>>guess I need to play more games, against a variety of programs to verify whether
>>>this improvement is real, or imaginary.
>>>
>>>Anyhow, beware of reading too much into ELO predictions of test suites..
>>>
>>>Cheers All,
>>>Tom
>>
>>Hi!
>>
>>Mr Irazoquis secret test-suite is very impressing! I think it´s about 111
>>positions. He can predict a new programs strength better than any other test I
>>have seen so far. If his predictions remains as good as his previous results, I
>>hope we can stop publishing our list and just play for fun.
>>
>>Bertil SSDF
>
> Why is this Test secret??
Because this guarantees that nobody will tune his program to do well in this
test.
I don't know how he does it exactly, but Enrique was able to predict the ratings
of the new SSDF list a few days before it was published, with a typical margin
of error of +/- 5 elo points...
Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.