Author: David Dahlem
Date: 14:45:49 06/15/04
Go up one level in this thread
On June 15, 2004 at 17:36:02, Dann Corbit wrote: >On June 15, 2004 at 17:29:39, David Dahlem wrote: > >>On June 15, 2004 at 17:16:14, Dann Corbit wrote: >> >>>On June 15, 2004 at 17:05:57, David Dahlem wrote: >>> >>>>On June 15, 2004 at 16:44:58, Dann Corbit wrote: >>>> >>>>>On June 15, 2004 at 16:00:08, David Dahlem wrote: >>>>> >>>>>>On June 15, 2004 at 15:54:23, Gian-Carlo Pascutto wrote: >>>>>> >>>>>>>On June 15, 2004 at 15:33:41, David Dahlem wrote: >>>>>>> >>>>>>>>One of the problems with the current method of testing engines with test suites >>>>>>>>(e.g. WM-Test) is the problem of proving that the proposed solution move is >>>>>>>>actually the best move, especially with positions of a positional nature. >>>>>>>>Perhaps a new method would avoid this problem, namely a suite of mate positions, >>>>>>>>with known, more easily proven solutions? Time to solution could be the criteria >>>>>>>>by which engines are evaluated. >>>>>>>> >>>>>>>>Just an idea. Any thoughts? Would this work? >>>>>>> >>>>>>>As long as the idea is to test matefinder speeds this is fine. >>>>>>> >>>>>>>Don't expect to get an indication to playing strength though. >>>>>>> >>>>>>>-- >>>>>>>GCP >>>>>> >>>>>>Well, this was just an idea, an unproven theory, but i would think some kind of >>>>>>formula could be developed, and i would also think stronger engines would score >>>>>>higher than weaker engines. :-) >>>>> >>>>>Probably they would. But what is the relationship? >>>>> >>>>>For instance, if I ride ten miles on my bike at 20 MPH, and I jog 5 miles down a >>>>>trail at 10 MPH, what is the conversion for benefit between the two forms of >>>>>exercise? >>>> >>>>Well, that's apples and oranges. More valid would be to time you on your bike to >>>>the finish line against someone elses time to the finish line. :-) >>> >>>That's my point. Both comparisons are apples to oranges. >> >>Comparison of elapsed time to the finish line over a certain distance between >>two competitors is like comparing apples and oranges? Then all horse races, >>vehicle races, etc. are meaningless? > >I take a horse and run him without a rider. Now, I am going to use this to >predict how he will run with a rider. Maybe there is a direct correlation, and >maybe there isn't. And if there is a direct correlation, what is it? > >A test suite does not predict how well an engine will play. If it did, then >Beowulf would beat Shredder 6, because Beowulf scored 288/300 on WAC at 5 >seconds, and Shredder 6 scored 285 (on a certain machine). Of course, Shredder >would pound the ever-loving stuffings out of Beowulf in actual game play. I agree totally, that's what got me thinking about test suites, and the reason i started this thread, hoping to start a dialog on better testing methods. Using mate problems may not be accurate enough either, but it seems to me that's a better method than using positions where the "best move" proposed is not always proven to be best. Regards Dave Regards Dave
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.