Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: test results

Author: Vincent Diepeveen

Date: 08:39:05 09/26/01

Go up one level in this thread


On September 26, 2001 at 11:19:04, Robert Hyatt wrote:

>On September 26, 2001 at 10:48:14, Vincent Diepeveen wrote:
>
>>On September 25, 2001 at 23:44:09, Robert Hyatt wrote:
>>
>>you didn't turn off futility bob and i cannot see outputs
>>in number of nodes and such to see whether sequential overhead
>>mattered!
>
>
>I don't do futility pruning.  I can't turn it off...
>
>
>
>>
>>Kopec positions are the worst testpositions ever to use anyway
>>for obvious reasons that the score only goes up and up and up.
>
>
>Not true.  Crafty changes its mind several times per test position...
>
>The data shows that the speedup is pretty constant regardless of null-move
>or not.  It would _also_ be constant with or without futility.  Older versions
>had futility pruning and/or razoring.  And it had no effect on the parallel
>search.
>
>If you want to make a wager, I will find the CB test positions and run
>them in the same way.  You will see _no_ change in my speedup results,
>however...

please run the positions and write down the speedups for 8,9,10,11,12 ply
of course for both 1 and 2 processors. 4 processors frommy viewpoint is
less interesting.

game positions are *better* than positions where a few strategical
principles dominate and where some of the positions are too easy
to be true. you only get a score higher and higher!

What you did here is compare apples with peanuts. depth 9 compared to
depth 10. That's stupid.

What we need is the crafty OUTPUT simply from each run at this game!

Easy to automate too.

>
>>
>>>Here are the results.  First the tests.  I took the last 8 kopec positions,
>>>and searched them to a fixed depth for each null-move setting.  9 plies for
>>>R=0, 10 plies for R=1, 11 plies for R=2 and 12 plies for R=2~3.
>>>
>>>I ran the tests with 1, 2 and 4 processors, and computed the speedup for
>>>each.  The data:
>>>
>>>null move R=0-----------------------------
>>>          1cpu          2cpu          4cpu
>>>pos17     115            67            40
>>>pos18     267           146            77
>>>pos19      61            32            17
>>>pos20     106            56            30
>>>pos21     126            71            36
>>>pos22     116            63            33
>>>pos23     108            59            31
>>>pos24     337           176            90
>>>  sum    1236           670           354
>>>  S/U     1.0           1.8           3.5
>>>
>>>
>>>null move R=1-----------------------------
>>>          1cpu          2cpu          4cpu
>>>pos17      42            22            15
>>>pos18      76            34            21
>>>pos19      32            16             9
>>>pos20      35            20            11
>>>pos21      30            15             9
>>>pos22      51            28            16
>>>pos23      68            36            19
>>>pos24     144            74            40
>>>  sum     478           245           140
>>>  S/U     1.0           1.9           3.4
>>>
>>>
>>>null move R=2-----------------------------
>>>          1cpu          2cpu          4cpu
>>>pos17      39            19            12
>>>pos18     121            55            18
>>>pos19      27            16             8
>>>pos20      34            19            13
>>>pos21      20            11             6
>>>pos22      43            22            12
>>>pos23      58            29            15
>>>pos24      83            44            28
>>>  sum     425           215           112
>>>  S/U     1.0           1.9           3.8
>>>
>>>
>>>null move R=2~3---------------------------
>>>          1cpu          2cpu          4cpu
>>>pos17      67            41            26
>>>pos18     265            99            60
>>>pos19      36            21            12
>>>pos20      90            52            27
>>>pos21      40            26            15
>>>pos22      74            41            21
>>>pos23     107            66            39
>>>pos24     194           106            51
>>>  sum     873           452           251
>>>  S/U     1.0           1.9           3.5
>>>
>>>
>>>
>>>The conclusions:
>>>
>>>1.  Crafty gets roughly 1.9X faster using two processors, regardless of
>>>the null-move setting.  R=0 (no null move at all) to r=2-3, the most
>>>aggressive setting I use.
>>>
>>>2.  It averaged a 3.5 speedup for 4 cpus, with R=2 having a slightly better
>>>speedup for random reasons.
>>>
>>>3.  Null-move has _zero_ influence on the speedup of a parallel search, as I
>>>have said _many_ times.  All this nonsense about saying that the old programs
>>>got better speedups without null-move, or better speedups with null-move is
>>>total baloney.
>>>
>>>Anybody else is free to run the same tests...  But I prefer to do things a
>>>bit scientifically by running a test, rather than wild speculation without
>>>any testing at all.
>>>
>>>I can provide the raw data if needed, but it would be a very large post since
>>>I ran the tests several times to average them.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.