Author: David Dahlem
Date: 15:42:15 06/15/04
Go up one level in this thread
On June 15, 2004 at 17:57:56, Dann Corbit wrote: >On June 15, 2004 at 17:45:49, David Dahlem wrote: > >>On June 15, 2004 at 17:36:02, Dann Corbit wrote: >> >>>On June 15, 2004 at 17:29:39, David Dahlem wrote: >>> >>>>On June 15, 2004 at 17:16:14, Dann Corbit wrote: >>>> >>>>>On June 15, 2004 at 17:05:57, David Dahlem wrote: >>>>> >>>>>>On June 15, 2004 at 16:44:58, Dann Corbit wrote: >>>>>> >>>>>>>On June 15, 2004 at 16:00:08, David Dahlem wrote: >>>>>>> >>>>>>>>On June 15, 2004 at 15:54:23, Gian-Carlo Pascutto wrote: >>>>>>>> >>>>>>>>>On June 15, 2004 at 15:33:41, David Dahlem wrote: >>>>>>>>> >>>>>>>>>>One of the problems with the current method of testing engines with test suites >>>>>>>>>>(e.g. WM-Test) is the problem of proving that the proposed solution move is >>>>>>>>>>actually the best move, especially with positions of a positional nature. >>>>>>>>>>Perhaps a new method would avoid this problem, namely a suite of mate positions, >>>>>>>>>>with known, more easily proven solutions? Time to solution could be the criteria >>>>>>>>>>by which engines are evaluated. >>>>>>>>>> >>>>>>>>>>Just an idea. Any thoughts? Would this work? >>>>>>>>> >>>>>>>>>As long as the idea is to test matefinder speeds this is fine. >>>>>>>>> >>>>>>>>>Don't expect to get an indication to playing strength though. >>>>>>>>> >>>>>>>>>-- >>>>>>>>>GCP >>>>>>>> >>>>>>>>Well, this was just an idea, an unproven theory, but i would think some kind of >>>>>>>>formula could be developed, and i would also think stronger engines would score >>>>>>>>higher than weaker engines. :-) >>>>>>> >>>>>>>Probably they would. But what is the relationship? >>>>>>> >>>>>>>For instance, if I ride ten miles on my bike at 20 MPH, and I jog 5 miles down a >>>>>>>trail at 10 MPH, what is the conversion for benefit between the two forms of >>>>>>>exercise? >>>>>> >>>>>>Well, that's apples and oranges. More valid would be to time you on your bike to >>>>>>the finish line against someone elses time to the finish line. :-) >>>>> >>>>>That's my point. Both comparisons are apples to oranges. >>>> >>>>Comparison of elapsed time to the finish line over a certain distance between >>>>two competitors is like comparing apples and oranges? Then all horse races, >>>>vehicle races, etc. are meaningless? >>> >>>I take a horse and run him without a rider. Now, I am going to use this to >>>predict how he will run with a rider. Maybe there is a direct correlation, and >>>maybe there isn't. And if there is a direct correlation, what is it? >>> >>>A test suite does not predict how well an engine will play. If it did, then >>>Beowulf would beat Shredder 6, because Beowulf scored 288/300 on WAC at 5 >>>seconds, and Shredder 6 scored 285 (on a certain machine). Of course, Shredder >>>would pound the ever-loving stuffings out of Beowulf in actual game play. >> >>I agree totally, that's what got me thinking about test suites, and the reason i >>started this thread, hoping to start a dialog on better testing methods. Using >>mate problems may not be accurate enough either, but it seems to me that's a >>better method than using positions where the "best move" proposed is not always >>proven to be best. > >I don't think it is better. During 90% or more of the moves in a chess game, >you will not be seeking a checkmate. You are striving to improve your position. > You are striving to win material. You are striving to put the enemy king into >peril (not necessarily a checkmate). You are striving to improve your pawn >formation. You are striving to produce a passed pawn. Checkmate tests do not >help in these areas, except by chance. > >Imagine the opening board. How will searching for the best checkmate in this >position make the program better? Clearly it won't have any bearing on program >strength. Yes, you are obviously quite right in all these points. By the way, what is the best checkmate from the opening position? :-) Regards Dave
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.