Author: Peter Fendrich
Date: 14:24:23 08/15/01
Go up one level in this thread
On August 15, 2001 at 13:08:12, Leen Ammeraal wrote: >I would suggest the following way to clean up this >test suite. Remove the following positions: > >1. those which are not solved even by a very strong chess engine > that takes a lot of time (position too difficult) >2. those which are easily solved by most chess engines, even > when they use little time (too easy) >3. those which show alternating results (position solved at > a certain depth but not solved at a greater depth) > with, say, two chess engines. > >Leen Ammeraal Maybe not remove but create different suites. It all depends on what you want to do with it. I would like to keep a big amount of "too easy" ones for debugging resons. After a change in the program and it doesn't find all the easy solutions something is definitely wrong... Under 1 we would have the wrong ones as well. It would be easier I think to identify them when group 1 is created. Heavy analyses and endless (in some cases!) discussions will follow... //Peter > >On August 15, 2001 at 12:59:21, Bruce Moreland wrote: > >>On August 15, 2001 at 06:52:18, Peter Kappler wrote: >> >>>On August 15, 2001 at 05:03:20, Peter Fendrich wrote: >>> >>>>Has anyone tried to clean up the ecm test suite? >>>>//Peter >>> >>> >>>There was a big effort here (or was it rgcc?) a few years ago. I don't know if >>>anybody went through the effort of verifying all the fixes and publishing a >>>"clean" version. I remember that many of the fixes/alternatives were very deep >>>variations and took a lot of CPU time to verify. >>> >>>Bruce Moreland has mentioned that he likes ECM -- he might have been part of >>>this effort. Probably Bob was in on the action, too. I can't recall the other >>>participants. >>> >>>It would be neat to try cleaning it up again. Machines are much faster now. :) >>> >>>-Peter >> >>The problem with that effort is that we didn't define what we were trying to do. >> Were we trying to cull bad positions? Were we trying to assure singular >>correctness of positions? Were we trying to assure that the key was the only >>move that won to some degree or another? Were we trying to find hard positions >>that were correct? >> >>There are lots of types of suites that would be of interest to people. A suite >>comprised of hard positions that can be solved with present techniques in >>between thirty seconds and an hour would be very useful, and would probably be >>useful for several more years. >> >>The problem with ECM is that it takes a long time to run to any appreciable >>depth, some of the problems are wrong, some have cooks, some end up being found >>for "positional" reasons, and many are way too easy. >> >>If I take the ECM positions that Ferret doesn't find in 20 seconds and make a >>suite out of them, I end up with a suite that almost by definition everyone else >>will do better on. >> >>I think it might make more sense to grade the positions somehow, and break the >>suite up into pieces that are easy or hard, get rid of the obvious mistakes, and >>get rid of the ones what cause positional arguments. >> >>I don't have time to do this now since my plane leaves tomorrow. >> >>bruce
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.