# Computer Chess Club Archives

## Messages

### Subject: Re: Proving something is better

Author: Bruce Moreland

Date: 12:32:55 12/18/02

Go up one level in this thread

```On December 18, 2002 at 12:13:56, Sune Fischer wrote:

>On December 18, 2002 at 11:38:41, Gian-Carlo Pascutto wrote:
>
>>On December 18, 2002 at 07:47:15, Sune Fischer wrote:
>>
>>>On December 17, 2002 at 19:42:10, Bruce Moreland wrote:
>>>
>>>>We have his new version, and it gets to the same depth more slowly, and finds
>>>>more answers, than R=3.  This proves nothing.  I could make a program where the
>>>>eval function incorporates a 2-ply search.  It would take longer to search 9
>>>>plies, but it would get a lot more right.  This is the same result that Omid
>>>>got.  Did he just prove that my hypothetical program is better?  Of course not.
>>>>
>>>>If you accept his method as proof, he did prove that VR=3 is better than R=2, I
>>>>point out.  But he should have tackled R=3, too, if he is going to present that
>>>>data.
>>>
>>>If you want to compare _search_ algorithms, you shouldn't go and change the
>>>evaluation or completely redefine the word "node" from one program to the next.
>>>
>>>The whole assumption here is that they are identical, except for changes in the
>>>search parameters.
>>
>>I don't see how this affects Bruce's point.
>>
>>His point is that searching slower to a certain depth but getting more
>>solutions is no proof that the algorithm is better.
>>
>>Are you arguing this is wrong?
>
>Yes I am.
>Bruce is saying that a node is a node.
>This is not the case if you pile a bunch of evaluation on some nodes and not on
>others, obviously those heavy nodes are going to a better job than the
>"lightweight" nodes.
>
>In _this_ case you can't compare nodes.
>
>A node is meant to be unit of search, change the unit and you change the
>everything.

I proposed a silly example, and you've gotten stuck on that.  I'll propose a
more sensible one and see where that gets us.

Take a program, call it A.  A uses R=2, and finds 50 solutions through ply 9.

Take another program, call it B.  B does not use null move at all, and it finds
55 solutions through ply 9.

Is B better than A?

You won't argue that B is better than A, because you'll tell me that of course B
takes much longer to produce its results.

But what you are telling me really is that you have some notions, and you are
willing to let these notions bridge a few gaps:  You believe that if they were
given equal time, that R=2 would find more than the version with no null move.

But my example doesn't say that.  For all you know, if you give them equal time,
the R=2 version will find 52 solutions.  The assumption that it will find (well)
in excess of 55 solutions is a leap of faith.

This is what is happening in Omid's paper.

R=3 finds 65 solutions in one unit of time.  R=2 finds 66 solutions in 2.2 units
of time.  Does this support the conventional logic that R=2 is better than R=3?
That would be a very poor leap of faith.

VR=3 finds 71 solutions 1.6 units of time.  VR=3 is shown to be better than R=2,
but is it shown to be better than R=3?  This is a better leap of faith, but it
is still a leap of faith.

bruce

```