Author: Mogens Larsen
Date: 10:19:07 07/19/00
Go up one level in this thread
On July 19, 2000 at 11:48:53, Robert Hyatt wrote: >Who makes that rule? I run engine vs engine tests all the time and use >ponder=on on a single machine with no problems at all. I use ponder=off >only when I am testing/debugging so I can get reproducible results when I >need them. You could do that, if you want to check all programs to make sure they use approx. 50% of the available cpu time. However, that applies to ponder off as well. I don't know if the loss in cputime is sufficiently compensated by extra time. You seem to have changed your mind about ponder=on on a single cpu machine if I recall correctly. My greatest concern is that with ponder on one machine you automatically exclude several programs without that feature, especially programs under development. Remember, when you call something "native" it has to be for objective reasons. Your reasons are purely subjective, because you're unable to detach yourself from the viewpoint of Crafty. >If you ask chess program users how they use their engines, 99.999% will say they >use it as it comes out of the box... which is "ponder=on". That is why I call >it "native mode". That is the _only_ way I play 'serious' games with Crafty >on the chess servers, at human events, etc. Yes, that's because they presume the opponent is a human. With engine-engine matches the native mode is ponder off on a single-cpu machine. At least that's my experience using the Fritz 6 demo. >All I can say, definitively, is that ponder=on has been tested for several >million games. Ponder=off has hardly been tested at all. I am happy with the >time usage in ponder=on games. I only know that it is tuned for that mode, and >that certain assumptions will fail if it is turned off. This is irrelevant, because it would be practically impossible to control that all features are tuned correctly in all programs tested. No matter if it's ponder=on or ponder=off. You're making a subjective point which is reasonable, but objectively irrelevant. >ponder=off is not intended for playing games, and I suppose I could add that >test to the code to not allow it. It is only intended for testing/debugging. >Have you ever used the "testev" command? Probably not as that is also a >debugging tool I use, but it is useless for anything else... Then why did you implement it in the first place? Did you assume that noone would use that option? That doesn't make sense. >I don't see an obligation for freeware authors _or_ commercial authors to do >enough testing to make sure that even oddball configurations work well. Ponder off is far from oddball. >I don't have the time to play as many games as needed to tune/test this. The >current time control code was developed over several years, with lots of tuning >and adjusting every few hundred games. If you really wanted to implement that feature correctly, I'm sure it would be possible to find a few people willing to help, myself included. Or you could ask someone. >I don't disagree. I simply state, after every such match I see, "this doesn't >necessarily reflect how these two engines would behave in a _real_ chess match >under normal circumstances (ponder=on, etc.)" I agree, there is very little data to suggest that, except Pittlik's data. In general I don't think there's any correlation between the comparable strength of programs using either ponder=on and off, except maybe in a small spectrum with certain conditions on timecontrol, hash, computer and whatever. My main objection is you complaining about data achieved by using a parameter included in the program, and writing it off as random. That's unappropriate IMHO, especially since the data put forward by Pittlik was interesting, but inconclusive by itself. There's no need to use the machinegun every time :o). Best wishes... Mogens
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.