Author: Robert Hyatt
Date: 10:30:39 01/19/98
Go up one level in this thread
On January 19, 1998 at 03:39:23, Don Dailey wrote: >On January 18, 1998 at 18:50:20, Amir Ban wrote: > >> >>Don, >> >>1. I'm ok with ECM98. >> >>2. Good to see general consensus on the data, but I think you are too >>eager to throw out on the basis of "flakiness". We really can't expect >>things to stabilize at ply 1, and when we have a move that is clearly >>better in the longer run, we shouldn't hesitate to say it is the >>solution (and to clearly say that any others are wrong). I don't know >>how many of the other 81 would pass your rigid criteria ... >> >>3. Who's coordinating this ? I nominate Bruce on the grounds that he has >>the best result so far. >> >>Amir > >Hi guys, > >I still had 302 positions that passed ALL my criteria. Your programs >would flunk out a few more but since everyone will disagree on easiness >we will get a few back. I think we could still end up with at least >200 that pass my "rigid" criteria. > >I won't complain if you want to include what I'm calling "flaky" >problems (as long as they are clearly best), but at least let me >make my case for being more selective before you outvote me! > >If we make an effort to have a clean set then: > > 1) You can shorten your test times by stopping on solution. > > 2) If you run to 2 minutes and stop you have to also either > examine each position or set up scoring information to > prove you would not have changed your mind had the set > continued. > > 3) Running the program longer should guarantee a better solution > count but this doesn't happen with flaky problems. > > >Please consider my proposal. If you don't like it, that's ok, I can >work with this kind of set too. > >- Don the only thing I want to see is a suite where each position has a clear solution, whether it be hard or easy, but where the solution move is not something that might show up "just because". IE the solution should be a tactical solution that requires a move that wouldn't normally be played, so we can compare apples to apples. The position Bruce mentioned from the BK test is but one example. Many people have, over the years, reported WAC results where they simply found the best move positionally without having any idea that it ends up winning material or mating. I'd like to see our suite produce unambiguous results... I'd like to suggest the following way to get this started: lets take the first 100 positions to begin with. If we all agree that 10 seconds is too easy, that's fine by me. If we'd rather look at them more carefully and eliminate the 3-4 ply solutions instead, it doesn't matter. But first, let's eliminate the easy ones. After we agree on those, then lets eliminate the ambiguous ones, or the ones with too many solutions, or the ones that we can't confirm that the solution is really best, etc... I will tally the results. I'd like to see Bruce do the same. Then we can compare notes to be sure neither missed a comment by someone. When we reduce the first 100 to N, then we move to the next 100. 100 seems managable, without producing huge posts. I'd also suggest we keep the FEN out of here. I have the entire suite already as does Bruce. It would save time/space to simply refer to the position by number, assuming we all have the same ECM suite from onenet. I'm going to check mine now and I will post a list of the ones I think are *not* too easy, to make my list short. We can "or" our lists if you'd like, and discuss why I like one and not another, and do the same for your results as well. I suspect the first 100 will go faster than the rest because I get 82 of them in 20 seconds, but only 569 of the whole set in that time. So I obviously miss many more later on...
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.