Author: Robert Hyatt
Date: 18:45:01 01/19/98
Go up one level in this thread
On January 19, 1998 at 18:04:29, Amir Ban wrote: >On January 19, 1998 at 13:30:39, Robert Hyatt wrote: > >>On January 19, 1998 at 03:39:23, Don Dailey wrote: >> > >> >>the only thing I want to see is a suite where each position has a clear >>solution, whether it be hard or easy, but where the solution move is not >>something that might show up "just because". IE the solution should be >>a tactical solution that requires a move that wouldn't normally be >>played, >>so we can compare apples to apples. The position Bruce mentioned from >>the >>BK test is but one example. Many people have, over the years, reported >>WAC results where they simply found the best move positionally without >>having any idea that it ends up winning material or mating. I'd like to >>see our suite produce unambiguous results... >> >> >>I'd like to suggest the following way to get this started: >> >>lets take the first 100 positions to begin with. If we all agree that >>10 seconds is too easy, that's fine by me. If we'd rather look at them >>more carefully and eliminate the 3-4 ply solutions instead, it doesn't >>matter. But first, let's eliminate the easy ones. After we agree on >>those, then lets eliminate the ambiguous ones, or the ones with too many >>solutions, or the ones that we can't confirm that the solution is really >>best, etc... >> >>I will tally the results. I'd like to see Bruce do the same. Then we >>can compare notes to be sure neither missed a comment by someone. When >>we >>reduce the first 100 to N, then we move to the next 100. 100 seems >>managable, >>without producing huge posts. > > >I don't object to your plan, but I have a slightly different target in >mind. I see more advantages than drawbacks in having a large suite. What >I don't want is to count one less when I should be counting one more >because the solution is not correct or there are others. I don't mind >the "flakes", and I think the "easy" ones have a place because this >means the suite has sensitivity to running at very short times. > >By all means go ahead with your plans, but my first priority is to get >the bugs out of the suite. > >Amir Suits me. How about we first fix the current ECM suite then. Either decide that a position needs an additional key move, or that a position should be discarded. I want to see the following: 1. no positions that are solved positionally at very shallow plies, because a program gets credit for playing the right move for the wrong reason. 2. no positions that have one key move given, but another key move produces an eval of +12 or something. I'd like to consider +12 as correct, unless we can demonstrate that +12 is wrong and a deeper search proves it.. 3. later I'd like to see easy ones discarded because running this entire thing for 1 or 2 minutes will take over half a day at one minute, over a full day at 2 minutes. That's too long...
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.