Author: Uri Blass
Date: 07:11:58 08/01/01
Go up one level in this thread
On July 31, 2001 at 23:19:48, Robert Hyatt wrote: >On July 31, 2001 at 21:38:58, Dieter Buerssner wrote: > >> >>Until now, I tried to trim the citations in my messages. It takes some editing. >>Nobody seems to care. Why? >> >>I think, if everybody would do this, we all could read this forum much more >>effectively. If only a minority is doing this, it is just work for this >>minority. I give up ... >> >>Then - you have stated many times, that engine matches on one computer make not >>much sense. Other people have stated, that it may make sense, and discussed, if >>ponder on or off would be more sensible. So, who is right? >> >>You more or less say, that the "one computer may get sensible results" fraction >>is wrong. Frank suggests an experiment (on 2 computers). You, from the beginning >>call this experiment invalid. > > > >Here is what I said, precisely: > >1. ponder=off is a poor way to play games. Because that is an unusual way to >run an engine and it is certainly possible (or even probable) that it is not >nearly so well-tested in that mode since it isn't used that way in serious >games. As a result, you are using the time-allocation code in a way it was >not well-tested, and perhaps in a way it was not designed to be used. The only >way to see is to look at the code, look at the games, and study how it uses time >with ponder=on and ponder=off to see if they are comparable. > >2. ponder=on makes more sense to me on a single machine, and I test like this >all the time. Both engines are 100% compute-bound, which means each machine >gets 1/2 the total processor time. It is like using two machines, with each >being 1/2 as fast as the actual machine. This might be unfair if you think that >one program plays better on faster hardware than it does on slower hardware, >when compared to the other program. Again you have to play matches both ways >and see if the results are comparable. > >Either way, you take a chance on producing results that are not comparable to >the results produced on two separate machines. Which begs the question "what >is the point?" > > > > > > > > > >> >>With the same right, the "one computer may get sensible results" fraction can >>say, that you should show evidence, that engine matches on one computer, with >>ponder off will show no reasonable results. So, they can ask you, to show an >>example, where it really makes a difference. How to solve this question? > >I won't take the time to find an example, any more than I will take the time >to carefully tune crafty's time allocation for ponder=off matches. It is time >that is wasted, and I don't have a lot of time to use in general.... > >I carefully (above) explained both options. Either one can produce flawed >results. I know that crafty runs better on faster hardware. So ponder=off >is better in that regard, perhaps. I know that crafty is not very good at >time management with ponder=off, so ponder=on is better in that regard, >definitely. Crafty generally predicts > 50% of the time correctly, which means >Frank's math is wrong and ponder=off does not make things significantly faster >than ponder=on. > >Two machines has _none_ of the above issues. Scientifically, it is the _right_ >way to play games. Why do you think all the commercial authors have machines >connected doing auto play all day long? Rather than using each machine to run >both programs at the same time with either ponder=off or on? > >Think about it... > > > > > >> >>You allways seem to assume, that all chess engines are only developed with >>ponder on in mind. I can say, that this is not true. Yace started with ponder >>off, and later I implemented a rather unsophisticated ponder mode. Yace doesn't >>have the puzzling mode, that Crafty has. The time management was written, while >>Yace had no pondering. I just added very few lines of code (certainly less then >>10), to make the engine use more time when pondering is on. So, perhaps, these >>two engines would be a good test for the question, how pondering affects engine >>strength. Especially, because our approach seems to be very different. I do not >>have 2 (reasonable) computers to play engine matches. Neither do I operate an >>account at some ICS myself. All my testing is with ponder off. >> > > >Have you never modified the time allocation code after watching it get into a >bit of time trouble with ponder=on? Or after seeing it reach a time control >with too much time left on the clock? > >How do you play games on the servers? On or off? > >How do you play test games to observe the program? On or off? > >I will bet the answers are yes, on and on... respectively. I guess that playing test games to observe yace are done usually with ponder off. Dieter explained that Fritz get most of the machine time if he tries to play engine-engine games against Fritz with ponder on so I guess that he plays these games with ponder off. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.