Author: Robert Hyatt
Date: 09:52:26 06/08/01
Go up one level in this thread
On June 07, 2001 at 18:04:10, Bruce Moreland wrote: >On June 07, 2001 at 16:27:20, Albert Silver wrote: > >>On June 07, 2001 at 13:14:32, Robert Hyatt wrote: >> >>>On June 07, 2001 at 12:33:44, Bruce Moreland wrote: >>> >>>>On June 07, 2001 at 10:52:33, Robert Hyatt wrote: >>>> >>>>>On June 07, 2001 at 06:41:44, Gian-Carlo Pascutto wrote: >>>>> >>>>>>On June 07, 2001 at 05:52:51, Jouni Uski wrote: >>>>>> >>>>>>>Yes I know all test suites are almost useless to finds the best playing chess >>>>>>>software. But still, which suite is the less bad in your opinion? If You can >>>>>>>only use one test suite before starting real games, which suite will you select? >>>>>>>Is it still LCT II? >>>>>> >>>>>>Following Bruce Moreland's recommendations, I'd have to name ECM. >>>>>> >>>>>>-- >>>>>>GCP >>>>> >>>>> >>>>>I don't like ECM as it has _way_ too many errors. The "modified" version >>>>>might be reasonable. >>>> >>>>I don't see why the errors matter. So I fail on those, big deal. If I run WAC, >>>>I get the same number every time. If I run ECM with more time, I'll find more >>>>solutions. To me this means that if I make the program better, I'll find more >>>>of them, which is what the suite is supposed to show. The suite has 879 >>>>positions. I think that well over 700 of them can be found, although I don't >>>>have an exact number. >>>> >>>>bruce >>> >>> >>>I don't like getting "better" and then getting more wrong answers. Or getting >>>worse and getting more "right" answers. The ones that are wrong are really >>>wrong. And it is certainly possible to make your program stronger and suddenly >>>start getting the wrong answers rather than the right ones. >>> >>>IE it is like grading a test with a key that randomly gets changed without >>>your knowing... >>> >>>I thought that was the reason we spent so much time going over the thing and >>>excluding obviously bad positions? >> >>Just out of curiousity, how long do the longest (and correct) solutions take? >> >> Albert > >Some of them take many hours and are similar in difficulty to the Nolot >problems. Unfortunately, the machine that has that info is unplugged and >downstairs, so I can't get at it. > >It wouldn't take geological time to figure out. Run them all at 20 seconds, >weed out the solved problems, run them at a minute, weed out the solved problem, >etc., until you are spending hours each. > >bruce I just ran a quick test using my quad xeon. At 10 seconds per move, I get this: search time per position...................10s total positions searched.......... 879 number right...................... 647 number wrong...................... 232 percentage right.................. 73 percentage wrong.................. 26 total nodes searched..............4029992645.0 average search depth.............. 7.3 nodes per second.................. 1221680 Some that were wrong had huge scores. IE +5 and yet the answer was something else. Some are probably really wrong as there may be a mate or even bigger win somewhere else. Some are probably really right as either the position was hosed when it was posted, or the solution has a "cook". I almost always run this at short time controls as it takes forever otherwise. I have no idea whether 647 right at 10 seconds is good, bad or horrible. I have gotten something close to that for a couple of years..
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.