Author: Howard Exner
Date: 13:42:29 04/30/98
Go up one level in this thread
On April 30, 1998 at 12:59:08, Bruce Moreland wrote: > >On April 30, 1998 at 07:57:44, Howard Exner wrote: > >>What time do the programmers want to use for deleting the >>easy ones? 10, 20, or 30 seconds (assuming like hardware)? > >Be very careful when you do this, and think about what you are doing. > >Here is what happened to me. > >I made a "hard" set out of ECM, I kept everything that wasn't solved in >under X seconds, although I don't remember what X was. > >Now, of course, what I have just done is created a test suite that my >program will perform worst on if it is run for under X seconds. I will >score zero on this suite, even a program that chooses randomly between >candidate moves will score more than this. > >And if the suite is run for longer than X seconds, other programs will >find some of the answers in less than X seconds, while mine won't find >any in that time. All of the problems that are easy for me and hard for >others have been deleted, leaving problems that might be easy for >others. > >So whatever programs you use to determine "easiness" are going to score >in last place when you run the suite for real. I follow your reasoning and example. This does pose a dilemma if just a few programs are chosen to delete ones that are common to them but as you say may not be easy for others. The new draft now has about 770 positions. Would it be possible to have various programs list the ones they find in under "X" then only delete the common ones? If this would work then it would be a matter of inviting programmers and avid testers alike to test machine X at say 10 seconds per move (factoring in hardware speed). Besides the original programmers who spearheaded this project I'll wager others would join in (other programmers and owners of programs).
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.