Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Bruce Moreland

Date: 15:55:35 12/17/02

Go up one level in this thread

On December 17, 2002 at 18:06:09, Dann Corbit wrote:

>Time to solution and number of solutions is the key for test problems (to my way
>of thinking).  If we count nodes, the slow searchers will look very bad.
>I think a bigger problem with test suites is that there is a plethora of
>excellent tactical test suites and a dirth of excellent positional test suites.
>So we measure tactical prowess effectively, but this does not necessarily
>translate into excellence in play.

Counting nodes means nothing if you are going to compare them between programs,
which Omid is not doing, so no knock on Omid there.

With this kind of thing, what you are trying to do is get (presumably) the same
result in less time.

Less nodes is not *quite* the same as less time, even with the same program.
It's like measuring someone's height by measuring their shadow.  It is more
direct to measure their height, even though measuring their shadow will get you

Getting to the same depth in shorter time is also not quite the same, if you
don't get at least as many problems correct.

Neither is getting to the same depth in more time and then saying this is better
because you find more solutions.

You have to compare apples with apples or you've *proven* nothing, all you've
done is *implied* something.  That's not science.

This has nothing to do with "excellence" in play.  The whole idea is to take
what excellences is there and make it faster.

Is this technique faster?  The data doesn't say.

And this is a chronic problem when people write articles on search techniques.


This page took 0.06 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.