Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Tony Werten

Date: 11:10:49 12/18/02

Go up one level in this thread

On December 18, 2002 at 03:58:31, Uri Blass wrote:

>On December 18, 2002 at 03:32:05, Ricardo Gibert wrote:
>>On December 18, 2002 at 02:28:29, Tony Werten wrote:
>>>On December 17, 2002 at 17:30:36, Bruce Moreland wrote:
>>>>Omid wrote an article that's in the ICGA that I received today, and there are
>>>>two tables:
>>>>D        R=1               R=2            R=3           VR=3
>>>>9     1,652,668,804    603,549,661   267,208,422    449,744,588
>>>>10   11,040,766,367  1,892,829,685   862,153,828  1,449,589,289
>>>>D        R=1               R=2            R=3           VR=3
>>>>9        64                62             53            60
>>>>10       71                66             65            71
>>>>The first table is nodes taken to search to depth D with four techniques, which
>>>>are standard null-move with various R values, and a "verified" null move search
>>>>with R=3.
>>>>The second table is the number of problem solutions in a test suite, given
>>>>particular depths of search and using these techniques.
>>>>The conclusion is that VR=3 is better.
>>>>Does anyone else see the two problems here?
>>>Yes, but 2 different ones then yours :)
>>>1)R2-3 is expected to search a number of nodes between r=2 and r=3, just as v3.
>>>Yet R2-3 is not in the table.
>>>2) Time to solution is missing (maybe the same as your point 2). I think that
>>>that table would be a good addition. Number of solutions at depth x would make
>>>the algoritm "extend every move at the root with 3 ply" look very good.
>>>I don't understand why time to solution is not used. The only argument I saw was
>>>"it's too hardware dependend". But I don't think that a good argument since a a
>>>percentile speedup ( ie 10% faster ) will still be the same no matter how much
>>>faster the hardware is.
>>If "better" means the TTS (=Time To Solution) for one algorithm is less than
>>anothers, then consider 2 different hardware architectures X and Y with respect
>>to algorithms A and B:
>>On Hardware X, the TTS for algorithm A may be less than the TTS for algorithm B.
>>Therefore algorithm A would be "better" than algorithm B.
>>Unfortunately, it is possible that on hardware Y, the TTS for algorithm A may be
>>greater than the TTS for algorithm B. In this case algorithm B would be "better"
>>than algorithm A. With this example, it is seen that this standard for "better"
>>leads to inconsistencies.
>>On the other hand if "better" means the NNS (=Number of Nodes Searched) for one
>>algorithm is less than anothers:
>>On hardware X, if the NNS of algorithm A is less than the NNS for algorithm B,
>>then algorithm A is "better" than algorithm B.
>>If we now switch to hardware B, the NNS of each algorithm will remain the same
>>of course, so the 2nd standard of "better" remains consistent, since it is
>>hardware independent. This is why NNS is preferred to TTS.
>It is dependent on the algorithm.
>If the algorithm use the time as a parameter in decisions about search then
>it is possible that you do not get the same results.
>Of course it is not relevant for comparison between verified null move pruning
>and null move pruning but you cannot generalize it for every 2 algorithms.

Maybe you can. I think that when NNS is used, any algoritm with more knowledge
will look best.

More knowledge gives either less nodes or solves a position at an earlier depth.

It doesn't matter if it takes you 2 hours to calculate that knowledge.



This page took 0.22 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.