# Computer Chess Club Archives

## Messages

### Subject: Re: Proving something is better

Author: Tony Werten

Date: 11:10:49 12/18/02

Go up one level in this thread

```On December 18, 2002 at 03:58:31, Uri Blass wrote:

>On December 18, 2002 at 03:32:05, Ricardo Gibert wrote:
>
>>On December 18, 2002 at 02:28:29, Tony Werten wrote:
>>
>>>On December 17, 2002 at 17:30:36, Bruce Moreland wrote:
>>>
>>>>Omid wrote an article that's in the ICGA that I received today, and there are
>>>>two tables:
>>>>
>>>>D        R=1               R=2            R=3           VR=3
>>>>9     1,652,668,804    603,549,661   267,208,422    449,744,588
>>>>10   11,040,766,367  1,892,829,685   862,153,828  1,449,589,289
>>>>
>>>>D        R=1               R=2            R=3           VR=3
>>>>9        64                62             53            60
>>>>10       71                66             65            71
>>>>
>>>>The first table is nodes taken to search to depth D with four techniques, which
>>>>are standard null-move with various R values, and a "verified" null move search
>>>>with R=3.
>>>>
>>>>The second table is the number of problem solutions in a test suite, given
>>>>particular depths of search and using these techniques.
>>>>
>>>>The conclusion is that VR=3 is better.
>>>>
>>>>Does anyone else see the two problems here?
>>>
>>>Yes, but 2 different ones then yours :)
>>>
>>>1)R2-3 is expected to search a number of nodes between r=2 and r=3, just as v3.
>>>Yet R2-3 is not in the table.
>>>
>>>2) Time to solution is missing (maybe the same as your point 2). I think that
>>>that table would be a good addition. Number of solutions at depth x would make
>>>the algoritm "extend every move at the root with 3 ply" look very good.
>>>
>>>I don't understand why time to solution is not used. The only argument I saw was
>>>"it's too hardware dependend". But I don't think that a good argument since a a
>>>percentile speedup ( ie 10% faster ) will still be the same no matter how much
>>>faster the hardware is.
>>
>>
>>If "better" means the TTS (=Time To Solution) for one algorithm is less than
>>anothers, then consider 2 different hardware architectures X and Y with respect
>>to algorithms A and B:
>>
>>On Hardware X, the TTS for algorithm A may be less than the TTS for algorithm B.
>>Therefore algorithm A would be "better" than algorithm B.
>>
>>Unfortunately, it is possible that on hardware Y, the TTS for algorithm A may be
>>greater than the TTS for algorithm B. In this case algorithm B would be "better"
>>than algorithm A. With this example, it is seen that this standard for "better"
>>
>>
>>On the other hand if "better" means the NNS (=Number of Nodes Searched) for one
>>algorithm is less than anothers:
>>
>>On hardware X, if the NNS of algorithm A is less than the NNS for algorithm B,
>>then algorithm A is "better" than algorithm B.
>>
>>If we now switch to hardware B, the NNS of each algorithm will remain the same
>>of course, so the 2nd standard of "better" remains consistent, since it is
>>hardware independent. This is why NNS is preferred to TTS.
>
>It is dependent on the algorithm.
>
>If the algorithm use the time as a parameter in decisions about search then
>it is possible that you do not get the same results.
>
>Of course it is not relevant for comparison between verified null move pruning
>and null move pruning but you cannot generalize it for every 2 algorithms.

Maybe you can. I think that when NNS is used, any algoritm with more knowledge
will look best.

More knowledge gives either less nodes or solves a position at an earlier depth.

It doesn't matter if it takes you 2 hours to calculate that knowledge.

Tony

>
>Uri

```