Author: Ulrich Tuerke
Date: 07:47:08 12/13/99
Go up one level in this thread
On December 12, 1999 at 05:45:01, Baldomero Garcia, Jr. wrote: >On December 11, 1999 at 21:14:14, Christophe Theron wrote: > >>> Why is this Test secret?? >> >> >>Because this guarantees that nobody will tune his program to do well in this >>test. > >I wasn't aware that programmers "tuned" their programs to do well in test >suites. In any case, if this tuning not only improves the scores in the test, >but also improves the actual performance of the program, then the test should be >available. I do not think that many "colleagues" intentionally tune their programs on suites. What I use to do when I'm producing a new version is that I run a set of purely tactical test positions consisting of some BT and BS entries and others, just to make sure that the new version does not loose the solution to one of the positions which my program usually solves. In case of some loss of this kind, I investigate if I will be going to live with this or I will modify the program again. This is of course "some kind of tuning" if you want so. So, a reliable and neutral test to evaluate programs should surely be kept secret. To your second question. In my experience, a tuned version of my program did never play better than the untuned version. Often, the opposite was true: a version doing MUCH better in BT tests used to loose against the original. Now, I am only looking to solve a minimum of purely tactical positions. I do not care about ELO numbers derived from test suites; I am afraid that's just too naive. Regards, Uli > >Baldo.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.