Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Some thoughts on engine vs. engine matches

Author: Robert Hyatt

Date: 06:26:19 06/28/99

Go up one level in this thread


On June 28, 1999 at 08:29:36, Francesco Di Tolla wrote:

>>I can only offer two data points.  In Crafty, _my_ timing allocation code
>>assumes that there will actually be more time to use than crafty has at the
>>point it has to make the decision.
>
>I'm sorry Bob, but I don't agree here. I think that if your program does this it
>is a "problem" of your program. Any program that supports a "ponder off" mode
>should also be able to play at maximum efficency within the time given.
>


Sorry but you are wrong.  Should I spend 1/2 my time testing with ponder=off
to be sure that it works optimally?  Or should I spend 100% of my time testing
in the way where the program plays the strongest?

ponder=off was done to allow me to do some testing on one machine with two
programs, but for _debugging_, not anything else.  If I thought that this mode
would be used to play 'real games' I would either test more (wasted time) or
eliminate ponder=off completely.  _either_ of those options would be bad IMHO...




>Of course if you have more time (while waiting for opponents move) you can play
>better, but in my personal opinion, if you assume that time is always there and
>speculate on this it is your fault.
>
>(This has nothing to do with trusting eng-eng matches, but is a general
>consideration).
>
>[...]
>
>>The other data point was an old Rebel.  Ed did all of his time setting while
>>"permanent brain" was being used.  When he turned it off in the NPS match he
>>saw bad time allocation too.
>
>again, something I would call "a bug" not a feature, or do newer version behave
>the same?
>
>[...]
>
>>Another point is that some programs (again, mine is an example) depends on
>>fairly fast hardware, because of some of the search decisions I have made in
>>the design process. [...]
>
>This is reasonable: tweaking a program for the hardware you run on is correct...
>
>>Your "ponder=off" type matches effectively cut the processor speed by 1/2,
>>when you think about it.  And the results can definitely be affected..
>
>... but this is not true, because you're looking at a different position, and
>even if you make assumptions on next move, you'll have a percentage of correctly
>guessed moves which is less than 100%, so the speed difference is less then one
>half.
>
>Moreover, is it better to compare two engines on two PC's with a P200 MMX (like
>SSDF fellows do) or let them coexist on a PII 400?
>Assuming this two cpu have a factor 2 of difference, would it be better to spend
>3 minutes on a move and 3 on the move of your opponent, or six on yours and none
>on the other?
>
>Finally, do the SSDF guys compare properly progrmas making them play on Intel's
>Pentiums when some engines (like Rebel) prefer other (AMD...).
>
>I have no real answer, so I think the important thing is not to take this
>comparisons to seriously, especially to don't buy after the positioning in the
>SSDF list!
>
>(Personally I choose the engine/program which gives me most fun when I play with
>it.)
>
>regards
>Franz



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.