Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Some thoughts on engine vs. engine matches

Author: Robert Hyatt

Date: 21:28:29 06/27/99

Go up one level in this thread


On June 27, 1999 at 21:51:52, Mark Young wrote:

>On June 27, 1999 at 21:15:23, Robert Hyatt wrote:
>
>>On June 27, 1999 at 19:46:43, Mark Young wrote:
>>
>>>On June 27, 1999 at 19:23:02, eric guttenberg wrote:
>>>
>>>>I can have Brett's opinion,too, and the evidence is abundant.  In the last
>>>>several weeks results of games between H7.32 and F5.32 on separate computers
>>>>have consistently shown the two programs to be close or have shown F5.32
>>>>to be better at blitz games; at the same time engine v engine results
>>>>have been posted over and over showing H7.32 to be almost 200 elo points
>>>>better than F5.32 at various time controls, including blitz. This difference
>>>>cannot be explained, except that engine v engine is a different animal than
>>>>computer v computer.
>>>>
>>>>eric
>>>
>>>My results on one computer have not shown this. I see this type of stuff all the
>>>time when some people post results by it on one computer or two. How knowns how
>>>they test or if they are showing the real and accurate results. I can only say I
>>>have seen nothing to say that you can not get accurate results with the
>>>chessbase interface playing it on one computer. If the data was scewed in a big
>>>way I would see it and it would not match up with my two computer results or
>>>other reliable two computer results.
>>
>>I can only offer two data points.  In Crafty, _my_ timing allocation code
>>assumes that there will actually be more time to use than crafty has at the
>>point it has to make the decision.  Because I _know_ that I will correctly
>>predict some moves here and there, and save that time.  And do I want to wait
>>until I save it before I use it, or should I use it in the part of the game
>>where it is important, rather than taking 10 minutes per move in a simple
>>endgame?
>>
>>The other data point was an old Rebel.  Ed did all of his time setting while
>>"permanent brain" was being used.  When he turned it off in the NPS match he
>>saw bad time allocation too.
>>
>>I don't know about others, but that is significant enough.
>
>Why is this significant enough, is fritz like this, is junior like this, etc. Is
>Hiarcs 7.32 like this. I am sorry I don't take one example and assume it applys
>to all.
>


I didn't give you _one_ example.  I gave you _two_.  And without more
information, the logical conclusion would be that 'ponder=off' probably
adversely affects _all_ programs.  Perhaps equally.  Perhaps not.  But if
two have shortcomings, it is certainly not illogical to think others might
also have problems.





>If we want to use crafty as an example...You know my results using crafty on one
>computer or two, or your computer. The results have been the same.
>


I don't have your data.  But I have many _other_ games vs hiarcs 7 that I
can supply.  Or you can find them on ICC with the 'search' command.  It
all depends on how 'stable' crafty happens to be at the time the games are
played.  I lose a bunch.  I win a bunch.  I'm always working on it...





>
>>
>>Another point is that some programs (again, mine is an example) depends on
>>fairly fast hardware, because of some of the search decisions I have made in
>>the design process.  Cut the speed by 1/2 and it might hurt me more than
>>another program.  Ask Thorsten about the Crafty vs CSTal match a couple of
>>years ago.  He was using p5/90's and Crafty was getting killed.  When he
>>went to something comparable to the P6/200, Crafty won way more than it lost.
>>
>>Your "ponder=off" type matches effectively cut the processor speed by 1/2,
>>when you think about it.  And the results can definitely be affected..



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.