# Computer Chess Club Archives

## Messages

### Subject: Re: Proving something is better

Author: Uri Blass

Date: 00:58:31 12/18/02

Go up one level in this thread

```On December 18, 2002 at 03:32:05, Ricardo Gibert wrote:

>On December 18, 2002 at 02:28:29, Tony Werten wrote:
>
>>On December 17, 2002 at 17:30:36, Bruce Moreland wrote:
>>
>>>Omid wrote an article that's in the ICGA that I received today, and there are
>>>two tables:
>>>
>>>D        R=1               R=2            R=3           VR=3
>>>9     1,652,668,804    603,549,661   267,208,422    449,744,588
>>>10   11,040,766,367  1,892,829,685   862,153,828  1,449,589,289
>>>
>>>D        R=1               R=2            R=3           VR=3
>>>9        64                62             53            60
>>>10       71                66             65            71
>>>
>>>The first table is nodes taken to search to depth D with four techniques, which
>>>are standard null-move with various R values, and a "verified" null move search
>>>with R=3.
>>>
>>>The second table is the number of problem solutions in a test suite, given
>>>particular depths of search and using these techniques.
>>>
>>>The conclusion is that VR=3 is better.
>>>
>>>Does anyone else see the two problems here?
>>
>>Yes, but 2 different ones then yours :)
>>
>>1)R2-3 is expected to search a number of nodes between r=2 and r=3, just as v3.
>>Yet R2-3 is not in the table.
>>
>>2) Time to solution is missing (maybe the same as your point 2). I think that
>>that table would be a good addition. Number of solutions at depth x would make
>>the algoritm "extend every move at the root with 3 ply" look very good.
>>
>>I don't understand why time to solution is not used. The only argument I saw was
>>"it's too hardware dependend". But I don't think that a good argument since a a
>>percentile speedup ( ie 10% faster ) will still be the same no matter how much
>>faster the hardware is.
>
>
>If "better" means the TTS (=Time To Solution) for one algorithm is less than
>anothers, then consider 2 different hardware architectures X and Y with respect
>to algorithms A and B:
>
>On Hardware X, the TTS for algorithm A may be less than the TTS for algorithm B.
>Therefore algorithm A would be "better" than algorithm B.
>
>Unfortunately, it is possible that on hardware Y, the TTS for algorithm A may be
>greater than the TTS for algorithm B. In this case algorithm B would be "better"
>than algorithm A. With this example, it is seen that this standard for "better"
>
>
>On the other hand if "better" means the NNS (=Number of Nodes Searched) for one
>algorithm is less than anothers:
>
>On hardware X, if the NNS of algorithm A is less than the NNS for algorithm B,
>then algorithm A is "better" than algorithm B.
>
>If we now switch to hardware B, the NNS of each algorithm will remain the same
>of course, so the 2nd standard of "better" remains consistent, since it is
>hardware independent. This is why NNS is preferred to TTS.

It is dependent on the algorithm.

If the algorithm use the time as a parameter in decisions about search then
it is possible that you do not get the same results.

Of course it is not relevant for comparison between verified null move pruning
and null move pruning but you cannot generalize it for every 2 algorithms.

Uri

```