Author: Ralf Elvsén
Date: 13:09:17 07/17/00
Go up one level in this thread
On July 17, 2000 at 15:39:41, Mogens Larsen wrote: >On July 17, 2000 at 14:48:37, Ralf Elvsén wrote: > >>If a program plays poorly with ponder = ON there is no excuse >>since this must be considered the default mode. > >No, that's not entirely true. That would depend on each program according to its >present state of development. Furthermore, default mode depends on the >configuration under which you're testing. Ponder on should be considered the >optimal or strongest setting. > >>What Bob has been >>trying to say is that he hasn't invested much time in ponder = OFF >>and so results from this way of matching computers >>aren't interesting (at least not to him). The question is not >>who understands pondering best. Often we don't >>know how much time an author has spent on tuning ponder = OFF, >>unless he states so clearly of course. > >Exactly, we don't know how much have been spent on either. So there's no reason >to assume that Bob has spent more time on ponder on or less time on ponder off >than anybody else. Claiming otherwise is a stab in the dark. What I mean is that ponder = ON is a feature that every program must have, at least if it wants to belong to the set of program whose strength can be assessed by playing matches. If a program performs poorly and the programmer came and said "I have only spent 5 minutes to implement time management with ponder = ON" , it would be a poor "excuse" since this is as natural as evaluation or whatever. So regardless of what we know about the programmers investment in this feature we can pit his program against opposition and get an accurate estimate. However, if we use ponder = OFF and don't know if this is supported we can't use these results with the same confidence since this is not something that is natural to do. It's another variable varied and we don't know in which direction. > >>Bob says it's not the >>way to play Crafty. One can't just dismiss him... > >Off course you can, but the results would be of no importance to Bob. >Considering that most people haven't got a dual or quad machine for ponder on >testing, it's important to acknowledge that the comparative strength between >programs appears to be almost unchanged by using ponder off. In principal I agree with Bob: matches with ponder = OFF should be looked upon with utter suspicion. However, as you say, there appears to be no significant difference. My opinion is this: the uncertainty because of ponder = OFF is probably not greater than the statistical uncertainty due to the small number of games, unless one plays hundreds and hundreds of games. So it probably doesn't matter which opinion you have, you don't know much anyway :) My own assessments of programs is a subjective weighting of numerical results and watching the games with PV:s and evaluations. Of course very few people would listen to this personal assessment but it's good enough for me :) And the subjective part is independent of the ponder issue... I would be glad if all programmers invested time in ponder = OFF and notified us. I doubt it will ever happen. Ralf > >Best wishes... >Mogens
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.