Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: The Stats just don't add up......

Author: Robert Hyatt

Date: 20:45:20 02/21/05

Go up one level in this thread


On February 21, 2005 at 21:08:05, Bryan Hofmann wrote:

>On February 20, 2005 at 19:56:51, Peter Skinner wrote:
>
>>On February 20, 2005 at 14:48:24, mike schoonover wrote:
>>
>>>hi all,
>>>been noticing this problem for a while with crafty.
>>>exits in ics mode quite freaquently.
>>>more with the newer ones.
>>>see:http://wbforum.volker-pittlik.name/viewtopic.php?t=1680
>>>it is not compile pessific.
>>>just wondering,is this a crafty or wb problem.
>>>help appreciated.
>>>regards
>>>mike
>>
>>I read your post on the wb forums, and the replies by Bryan Hoffman.
>>
>>I decided to test his version vs mine on the same computer, as I haven't done it
>>in a while.  Here are the results:
>>--------------------------------------------------------------------------------
>>unable to open book file [./book.bin].
>>book is disabled
>>unable to open book file [./books.bin].
>>hash table memory =   24M bytes.
>>pawn hash table memory =    6M bytes.
>>
>>Crafty v19.19 BH
>>
>>White(1): bench
>>Running benchmark. . .
>>......
>>Total nodes: 89729038
>>Raw nodes per second: 766914
>>Total elapsed time: 117
>>SMP time-to-ply measurement: 5.470085
>>White(1):
>>
>>EPD Kit revision date: 1996.04.21
>>unable to open book file [./book.bin].
>>book is disabled
>>unable to open book file [./books.bin].
>>hash table memory =   24M bytes.
>>pawn hash table memory =    6M bytes.
>>
>>Crafty v19.19 (1 cpus)
>>
>>White(1): bench
>>Running benchmark. . .
>>......
>>Total nodes: 89729038
>>Raw nodes per second: 787096
>>Total elapsed time: 114
>>SMP time-to-ply measurement: 5.614035
>>White(1):
>>-------------------------------------------------------------------------------
>>
>>Mine still seems slightly faster, and is the default compile with VC++ 2005
>>Express.
>>
>>Using these options:
>>
>>cl /Ox /O2 /GL /Gs /GA /GF /GT /Gr /MT /w /DNT_i386 /DWIN32 /D_CONSOLE /DWINDOWS
>>/DFAST /DEGTB6 /DEPD /DFUTILITY /DVC_INLINE_ASM crafty.obj egtb.obj
>>
>>Peter
>
>I find this hard to believe as I just ran your compile vs my compile on two
>different systems and Intel Pentium 4 2.8 GHz and a AMD 3000+ XP system. I'm
>using the full VC 2005 with POGO. The largest difference is in the AMD and I
>sure this is due to the POGO is being done on the AMD system.
>
>
>AMD 3000+ XP
>
>Skinners
>Crafty v19.19
>
>White(1): ben
>Running benchmark. . .
>......
>Total nodes: 96761642
>Raw nodes per second: 1256644
>Total elapsed time: 77
>SMP time-to-ply measurement: 8.311688
>White(1): quit
>
>Crafty v19.19 BH
>
>White(1): ben
>Running benchmark. . .
>......
>Total nodes: 96761642
>Raw nodes per second: 1362840
>Total elapsed time: 71
>SMP time-to-ply measurement: 9.014085
>
>
>
>
>Pentium 4 2.8GHz
>
>Skinners
>EPD Kit revision date: 1996.04.21
>unable to open book file [./book.bin].
>book is disabled
>unable to open book file [./books.bin]
>
>Crafty v19.19
>
>White(1): ben
>Running benchmark. . .
>......
>Total nodes: 96761642
>Raw nodes per second: 1018543
>Total elapsed time: 95
>SMP time-to-ply measurement: 6.736842
>White(1): quit
>
>Mine
>unable to open book file [./book.bin].
>book is disabled
>unable to open book file [./books.bin]
>
>Crafty v19.19 BH
>
>White(1): ben
>Running benchmark. . .
>......
>Total nodes: 96761642
>Raw nodes per second: 1063314
>Total elapsed time: 91
>SMP time-to-ply measurement: 7.032967
>White(1):


Hate to tell you guys, but you are all pissin' in the wind.  :)

optimizations change from one processor to another, and I am not just talking
about AMD vs Intel.  Different memory timing, different cache size/timing,
different memory latency, different processor timings, the list goes on and on,
and each can affect the speed of the program sporadically and unpredictably.
Even poor memory/cache aliasing can make the same executable vary in speed
significantly from one day to the next on the same processor.

Benchmarking and optimizing is not a "compile one time, run one test, and look
at the results".  It is a "compile once, run a bunch of tests, then clear memory
and run the same thing again.  Multiple times...  Whether you average or use the
best/worst/typical result is up to you, but there are too many variables for one
person to compile and think "this is the best there is".  There are even
compiler and optimizer differences to contend with beyond hardware difference...



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.