Author: Vincent Diepeveen
Date: 11:37:15 09/04/02
Go up one level in this thread
On September 04, 2002 at 14:18:38, Vincent Diepeveen wrote:
another problem of your measuring speedups is that you cannot
compare your own speedups on different positions even. There
are 2 reasons:
a) the first position you start with a cleaned hashtables the
others you do not
b) positions where previous move you mispredicted the move
the speedup is not comparable with predicted moves. In fact
it's unclear even whether you add to it the time you searched
in permanent brain.
c) because of the opponent move 1 gets played after x time, move 2
after y time. In short it is not possible to make a scientific
compare from it
>On September 04, 2002 at 13:36:38, Robert Hyatt wrote:
>
>But you make stupid assumptions. For example, DIEP profits about
>for 40% of the nodes in such a case. Imagine it profits 40%.
>
>That's dual with double node count than single. So run A it runs
>with n nodes a second. run B it runs with n/2 nodes a second.
>
>It means a speedup at 2 processors of > 2.0 is very well possible.
>It simply says nothing on how a program lineairly scales. the
>'speedup' number is no longer a speedup number but an index which
>is comparable with not a single scientists result.
>
>If your goal is to write somethign that cannot be compared to any other
>research, then i would love to hear it.
>
>>On September 04, 2002 at 13:23:08, Vincent Diepeveen wrote:
>>
>>>On September 04, 2002 at 13:10:25, Robert Hyatt wrote:
>>>
>>>No one uses these speedup numbers except you bob.
>>
>>So? The numbers are more meaningful, if they are harder to produce,
>>that's just a problem you have to overcome. But Crafty plays games. It
>>is interesting to know how it behaves in "near game" conditions, which are
>>closer to how it is used than a random set of disconnected positions from
>>all over...
>>
>>
>>>
>>>Not clearing the hashtable is not a fair way to test,
>>>but of course a great way to get a better speedup than
>>>you actually get.
>>
>>that last sentence is unparsable by me... "get a better speedup than you
>>actually get" can't be parsed by me or any software I have access to.
>>
>>not a fair way to test? What is a chess engine written to do? Solve
>>disconnected positions? Or play a complete game. As the DTS article
>>said, the reason I tested that was was because I was _continually_ asked
>>"how does the 16 cpu machine speed up your program compared to a one-cpu
>>machine, in games?" I couldn't answer because my PhD dissertation was
>>based on the kopec 24 positions (23 actually, I deleted #1 as useless).
>>
>>So I set out to answer _that_ question. And clearly in real games I
>>don't clear the hash table. So the test is _perfectly_ valid. Since
>>most are playing games and not solving randomly-selected positions.
>>
>>>
>>>Also we watn the node counts obviously and as crafty is
>>>one of the few with a asymmetric king safety it's also
>>>better to turn that off by using the 'analyze' feature.
>>>
>>
>>Why? Shouldn't the speedup be the actual numbers produced in a game?
>>
>>Aha. you want to finagle around and find the settings that make the
>>speedups the _worst_ so that you can compare to those?
>>
>>The DTS article was _clear_ about its intent of finding the speedup during
>>a game rather than for disconnected positions. "during a game" is the key
>>point. And a game is not played with asymmetry off if the program normally
>>has it on. Test your program however you want. But don't try to tell me
>>why _my_ testing positions and methodology are no good. They are at _least_
>>as good as yours. After all, I am not the one claiming super-linear speedup
>>is a normal occurrence. My tests don't show that at all, except for an
>>occasional random position...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.