Author: Dann Corbit
Date: 12:11:22 02/10/99
Go up one level in this thread
On February 10, 1999 at 14:25:36, Bruce Moreland wrote: > >On February 10, 1999 at 12:53:32, Dann Corbit wrote: > >>Don't you think that finding out which positions are broken in the commonly >>available test suites is valuable? I have found errors in every test suite. >>These test suites are used to judge the strength of your program. The positions >>that I posted were classified as "unsolved" because (up to that point) the >>results were unknown. By that I mean, I had already run them for a very long >>time, and the proposed move was not chosen, and there is no result of mate for >>either side. >> >>I think that finding the right answers or discovering what is wrong with the >>test problems is a lot more interesting than just finding an answer we already >>know. >> >>But I have an unusual bent, anyway. > >This is going to sound like a slam, but it's not. > >Personally, I have no interest in debugging chess books, which is what ECE >amounts to. The ECE3 test has been around for years, and I doubt there is much >if any interest in revising it. It is very large and I don't think it's used >much. > >"Here is a set of problems that are hard" is something I will respond to. >That's interesting. I like hard problems, I like seeing who gets them how fast, >to a point, and I enjoy seeing cases where chess knowledge, or lack of it, is at >issue. > >"Here is a set of problems where the published solution is in error" is >something I will rarely respond to. If the position is particularly famous or >the cook is interesting, sure. But these seem to just be mistakes, although I >did enjoy the one where black is mated after 1. ... g6. > >Similarly with the "hard mates" issue. To me that implies middlegame positions >with violent tactical solutions, or at least endgame positions with enough >pieces that you have to find the solution with search. It's also nice if the >position is somehow noteworthy, for instance almost every endgame study >qualifies. In those if I come back at -5 I know that I missed something, I >don't have to worry as much that the composer/annotator was in a hurry and >dropped a rook. > >I don't think a mate in 37 KR vs KN is interesting at all. You could make a >test suite consisting of thousands of problems of that type, taken more or less >at random from endgame database. All those are good for is checking to see if >you have endgame tables installed. Perhaps it would be useful in some cases to >see if a program can find "only" moves with search, but that would take a lot >more work on the part of the suite builder. > >I am not the test suite cop, obviously. Post whatever you want. Please >describe the suites accurately and I'll run the ones that I think are useful. I will try to be more accurate in descibing any problem set that I do post. I do not consider your statements a slam. In fact, I have never (even once) seen you offer any criticism that was not constructive. Your suggestion to describe my requests more fully is an excellent one that I plan to follow through on. Eventually, I will be able to produce exactly the suite you are interested in. For instance, I can find problems that took x number of plies to solve, for instance, fairly easily. I have all the data in a SQL database. I can also create new test positions by examining large swings in ce that were discovered by extending the depth of a search to great levels. The problem set I am working on right now may or may not have solutions to the problems at hand. I think (though I have not made a certain tally) that each and every EPD suite has mistakes in it. Certainly most of them do, some more than others (obviously). Since these test suites are used to judge the strength of your programs, it seems that programmers might take an interest.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.