Author: Dann Corbit
Date: 14:57:56 06/15/04
Go up one level in this thread
On June 15, 2004 at 17:45:49, David Dahlem wrote: >On June 15, 2004 at 17:36:02, Dann Corbit wrote: > >>On June 15, 2004 at 17:29:39, David Dahlem wrote: >> >>>On June 15, 2004 at 17:16:14, Dann Corbit wrote: >>> >>>>On June 15, 2004 at 17:05:57, David Dahlem wrote: >>>> >>>>>On June 15, 2004 at 16:44:58, Dann Corbit wrote: >>>>> >>>>>>On June 15, 2004 at 16:00:08, David Dahlem wrote: >>>>>> >>>>>>>On June 15, 2004 at 15:54:23, Gian-Carlo Pascutto wrote: >>>>>>> >>>>>>>>On June 15, 2004 at 15:33:41, David Dahlem wrote: >>>>>>>> >>>>>>>>>One of the problems with the current method of testing engines with test suites >>>>>>>>>(e.g. WM-Test) is the problem of proving that the proposed solution move is >>>>>>>>>actually the best move, especially with positions of a positional nature. >>>>>>>>>Perhaps a new method would avoid this problem, namely a suite of mate positions, >>>>>>>>>with known, more easily proven solutions? Time to solution could be the criteria >>>>>>>>>by which engines are evaluated. >>>>>>>>> >>>>>>>>>Just an idea. Any thoughts? Would this work? >>>>>>>> >>>>>>>>As long as the idea is to test matefinder speeds this is fine. >>>>>>>> >>>>>>>>Don't expect to get an indication to playing strength though. >>>>>>>> >>>>>>>>-- >>>>>>>>GCP >>>>>>> >>>>>>>Well, this was just an idea, an unproven theory, but i would think some kind of >>>>>>>formula could be developed, and i would also think stronger engines would score >>>>>>>higher than weaker engines. :-) >>>>>> >>>>>>Probably they would. But what is the relationship? >>>>>> >>>>>>For instance, if I ride ten miles on my bike at 20 MPH, and I jog 5 miles down a >>>>>>trail at 10 MPH, what is the conversion for benefit between the two forms of >>>>>>exercise? >>>>> >>>>>Well, that's apples and oranges. More valid would be to time you on your bike to >>>>>the finish line against someone elses time to the finish line. :-) >>>> >>>>That's my point. Both comparisons are apples to oranges. >>> >>>Comparison of elapsed time to the finish line over a certain distance between >>>two competitors is like comparing apples and oranges? Then all horse races, >>>vehicle races, etc. are meaningless? >> >>I take a horse and run him without a rider. Now, I am going to use this to >>predict how he will run with a rider. Maybe there is a direct correlation, and >>maybe there isn't. And if there is a direct correlation, what is it? >> >>A test suite does not predict how well an engine will play. If it did, then >>Beowulf would beat Shredder 6, because Beowulf scored 288/300 on WAC at 5 >>seconds, and Shredder 6 scored 285 (on a certain machine). Of course, Shredder >>would pound the ever-loving stuffings out of Beowulf in actual game play. > >I agree totally, that's what got me thinking about test suites, and the reason i >started this thread, hoping to start a dialog on better testing methods. Using >mate problems may not be accurate enough either, but it seems to me that's a >better method than using positions where the "best move" proposed is not always >proven to be best. I don't think it is better. During 90% or more of the moves in a chess game, you will not be seeking a checkmate. You are striving to improve your position. You are striving to win material. You are striving to put the enemy king into peril (not necessarily a checkmate). You are striving to improve your pawn formation. You are striving to produce a passed pawn. Checkmate tests do not help in these areas, except by chance. Imagine the opening board. How will searching for the best checkmate in this position make the program better? Clearly it won't have any bearing on program strength.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.