Author: Don Dailey
Date: 11:01:57 04/30/98
Go up one level in this thread
On April 30, 1998 at 12:59:08, Bruce Moreland wrote: > >On April 30, 1998 at 07:57:44, Howard Exner wrote: > >>What time do the programmers want to use for deleting the >>easy ones? 10, 20, or 30 seconds (assuming like hardware)? > >Be very careful when you do this, and think about what you are doing. > >Here is what happened to me. > >I made a "hard" set out of ECM, I kept everything that wasn't solved in >under X seconds, although I don't remember what X was. > >Now, of course, what I have just done is created a test suite that my >program will perform worst on if it is run for under X seconds. I will >score zero on this suite, even a program that chooses randomly between >candidate moves will score more than this. > >And if the suite is run for longer than X seconds, other programs will >find some of the answers in less than X seconds, while mine won't find >any in that time. All of the problems that are easy for me and hard for >others have been deleted, leaving problems that might be easy for >others. > >So whatever programs you use to determine "easiness" are going to score >in last place when you run the suite for real. > >bruce Bruce and Howard: Bruce is right about this. I think we definitely want to eliminate the trivial ones but we should be careful how we define trivial. Trivial for your program may not be challenging for mine. I would suggest using a little human judgement here. Perhaps if many programs solve it very quickly and it's easy to see that the solution involves trivial tactics and not any significant position judgment then it could be thrown out. I like using the model of how many ply a classical program would take. 5 or less might be considered trivial. I define "classical program" as one with only check extensions, full width, no selectivity and typical quies search (no cks in quies etc.) A very simple model and it's easy to calculate how many ply a purely tactical position should take to solve. However even this is not purely trivial because some programs can choke in the selective portion. A classic 5 ply problem could turn out to be several more in the worst case, trapped pieces, long term threats etc. I would recommend only punting the really easy ones as measured unanimously by several programs. It won't hurt to have a few that solve quickly, the testing time will also be trivial, especially if we know we can stop on solution due to clean problems. - Don
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.