Author: Robert Hyatt
Date: 11:02:00 07/19/00
Go up one level in this thread
On July 19, 2000 at 13:19:07, Mogens Larsen wrote: >On July 19, 2000 at 11:48:53, Robert Hyatt wrote: > >>Who makes that rule? I run engine vs engine tests all the time and use >>ponder=on on a single machine with no problems at all. I use ponder=off >>only when I am testing/debugging so I can get reproducible results when I >>need them. > >You could do that, if you want to check all programs to make sure they use >approx. 50% of the available cpu time. However, that applies to ponder off as >well. I don't know if the loss in cputime is sufficiently compensated by extra >time. You seem to have changed your mind about ponder=on on a single cpu machine >if I recall correctly. > Not that I recall. First, I don't like single-cpu tests, for anything other than what I use them for, namely debugging. If you insist on single-cpu tests, then I still prefer native-mode ponder=on. Ponder=off actually is worse with some programs. IE older chessmaster programs would use 100% of the cpu even with ponder=off. Because of a keyboard input loop. So in a ponder=off match, it would get roughly 75% of the total cpu cycles. Ponder=on at least makes that 'equal' assuming the operating system is sane about process scheduling. But for real tests, two machines is the only way to go, because that is the way real games are going to be played... IE a match between a computer and an IM in a shopping mall is interesting, but it won't show how the same match would turn out in a quiet tournament hall. Oddball conditions are ok, if you want to know how programs do under oddball conditions. But to extrapolate from that to normal conditions is a real stretch of the imagination... all IMHO of course.. >My greatest concern is that with ponder on one machine you automatically exclude >several programs without that feature, especially programs under development. > >Remember, when you call something "native" it has to be for objective reasons. >Your reasons are purely subjective, because you're unable to detach yourself >from the viewpoint of Crafty. > >>If you ask chess program users how they use their engines, 99.999% will say they >>use it as it comes out of the box... which is "ponder=on". That is why I call >>it "native mode". That is the _only_ way I play 'serious' games with Crafty >>on the chess servers, at human events, etc. > >Yes, that's because they presume the opponent is a human. With engine-engine >matches the native mode is ponder off on a single-cpu machine. At least that's >my experience using the Fritz 6 demo. > >>All I can say, definitively, is that ponder=on has been tested for several >>million games. Ponder=off has hardly been tested at all. I am happy with the >>time usage in ponder=on games. I only know that it is tuned for that mode, and >>that certain assumptions will fail if it is turned off. > >This is irrelevant, because it would be practically impossible to control that >all features are tuned correctly in all programs tested. No matter if it's >ponder=on or ponder=off. You're making a subjective point which is reasonable, >but objectively irrelevant. > >>ponder=off is not intended for playing games, and I suppose I could add that >>test to the code to not allow it. It is only intended for testing/debugging. >>Have you ever used the "testev" command? Probably not as that is also a >>debugging tool I use, but it is useless for anything else... > >Then why did you implement it in the first place? Did you assume that noone >would use that option? That doesn't make sense. > >>I don't see an obligation for freeware authors _or_ commercial authors to do >>enough testing to make sure that even oddball configurations work well. > >Ponder off is far from oddball. > >>I don't have the time to play as many games as needed to tune/test this. The >>current time control code was developed over several years, with lots of tuning >>and adjusting every few hundred games. > >If you really wanted to implement that feature correctly, I'm sure it would be >possible to find a few people willing to help, myself included. Or you could ask >someone. > >>I don't disagree. I simply state, after every such match I see, "this doesn't >>necessarily reflect how these two engines would behave in a _real_ chess match >>under normal circumstances (ponder=on, etc.)" > >I agree, there is very little data to suggest that, except Pittlik's data. In >general I don't think there's any correlation between the comparable strength of >programs using either ponder=on and off, except maybe in a small spectrum with >certain conditions on timecontrol, hash, computer and whatever. > >My main objection is you complaining about data achieved by using a parameter >included in the program, and writing it off as random. That's unappropriate >IMHO, especially since the data put forward by Pittlik was interesting, but >inconclusive by itself. There's no need to use the machinegun every time :o). > >Best wishes... >Mogens
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.