Author: Ralf Elvsén
Date: 14:26:37 07/17/00
Go up one level in this thread
On July 17, 2000 at 16:57:49, Mogens Larsen wrote: >On July 17, 2000 at 16:09:17, Ralf Elvsén wrote: > >>What I mean is that ponder = ON is a feature that every program must >>have, at least if it wants to belong to the set of program whose >>strength can be assessed by playing matches. > >Not true. In most matches the hardware available would be unable to use ponder >on without skewing the results significantly. It's very relevant against human >opposition, but that is a minor, although important, special case. We obviously have different opinions on in which settings the most interesting results can be found. I think human opposition or two-comp games are most important. You think one-comp games. They may be more frequent here, but the people posting here are a little special :) It doesn't take a lot of effort and money to arrange comp-comp games if one really wants to. Human opposition is the ultimate testground for many years to come. > >>If a program performs poorly >>and the programmer came and said "I have only spent 5 minutes to implement >>time management with ponder = ON" , it would be a poor "excuse" since this >>is as natural as evaluation or whatever. So regardless of what we know >>about the programmers investment in this feature we can pit his program >>against opposition and get an accurate estimate. However, if we use >>ponder = OFF and don't know if this is supported we can't use these >>results with the same confidence since this is not something that is >>natural to do. It's another variable varied and we don't know in which >>direction. > >Most, if not all, programs starts as ponder off programs only, so I think you're >ignoring how a program is developed. AFAIK, ponder off is always supported. At >least that's the experience I have from the programs I've tested where I've been >in contact with the programmer. It's the easiest way to test for bugs against >another program on one machine. Of course ponder OFF is supported and ponder ON is almost equally trivial to implement. The existence of these features, however, has nothing to do with how well they are supported. Time management is a complex affair and a programmer must invest a lot of time to solve it. So with minimal effort both modes can be supported in the sense that they exist. Then comes the tricky part of doing it good. The programmer chooses: do I implement both or one, and if so which one? Maybe some/many of the amateur programmers concentrate on ponder = OFF. One thing must be certain though: the professionals make sure their ponder = ON management is as good as it can be. How much time they spend on ponder = OFF is often completely unknown. And one professional, Bob, doesn't spend any time at all after his rudimentary implementation. He has stated as much that it takes endless hours of testing to get it right and it's not worth the effort. I suspect there are all kinds between the extremes. That ponder = OFF comes first and is good to find bugs has nothing to do with its quality. As far as I can tell, time management and search/eval bugs are orthogonal creatures mostly (but hash tables is a counter example). > >>In principal I agree with Bob: matches with ponder = OFF should be >>looked upon with utter suspicion. > >There's no reason for suspicion, because the same arguments applies to ponder on >without taking statistics into consideration. The games with ponder are most >likely better, because more time is available, but testing isn't about >esthetics. No, the same argument doesn't apply. Show me one programmer who thinks ponder = ON isn't an essential part of a chess program. > >>I would be glad if all programmers invested time in ponder = OFF and >>notified us. I doubt it will ever happen. > >Most of them invest time in ponder off and when that works properly they >implement ponder on. There's no need for further development. Fritz 6 can play >without ponder and I think that applies for all programs. > >Best wishes... >Mogens Best whishes to you as well Ralf
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.