Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Peter McKenzie

Date: 11:21:50 12/18/02

Go up one level in this thread

On December 18, 2002 at 06:19:35, Ricardo Gibert wrote:


Thanks for your reply.

>You're taking a purely practical point of view.


>That's fine, but scientific
>journals put great weight on theory.

What exactly is the theory of computer chess??  Sure we have a bunch of
algorithms and heuristics but we don't have any laws or axioms or anything like
that.  Computer chess is basically a part of behavioural AI, where behaviour is
what counts.

So I submit that computer chess is a field where practical considerations
dominate.  The most important thing is what works best in practice, not what has
the best theoretical properties.  Sure you can define certain interesting
theoretical properties, search tree size for a search for example, but they are
not the primary focus.

A Search technique like nullmove is just a tradeoff, there is actually NO THEORY
that describes why that particular tradeoff works better than another approach.
It all comes down to empirical testing, so lets do that in a useful way.

>>To put it another way, I think thing your argument is invalid and here is why:
>>We have 2 definitions of 'better'.
>My "argument" can't be invalid. I was explaining "why" NNs is often used in a
>journal like ICGA. You can disagree with the reason "why" if you wish and your
>reasons for disagreeing are indeed good ones, but the "why" still remains.

You are right, your explanation was perfectly valid.  I think I shot the
messenger :-)

It is the "why" which I take issue with.

>>One of them, based on TTS means that the algorithm is definitely better for some
>>real computer system.  Yes, there actually exists a computer where we are 100%
>>sure that this algorithm is superior for a real world applicaton.  Is it
>>superior on all computers?  Well, we don't know for sure although it may be
>>possible to draw some inference.
>I don't disagree with this, but you are ignoring *one* of the aims of journals
>like ICGA which is to enlarge the body *theory* for the field the journal
>concerns itself with.

I'm don't think I'm ignoring that.  The journal should enlarge the body of
theory, but in doing so it is usually useful to link that theory to practice.

>For example, computer science journals will sometimes have an article that has
>an algorithm that is in theory "better" than all the others, but which has
>absolutely no practical value whatsoever. Such articles do not interest me and
>they probably do not interest you, but there they are.

I certainly have no problem with publishing novel algorithms which are worse
than the known algorithms.  This may stimulate thought and lead to better

>Please note that I suggested using both NNS and TTS further down. Now how bad
>can that be?

Right, that is the ideal.

>>Now, take the definition of 'better' based on NNS.  We don't actually know if
>>this algorithm will perform better for a real world application on any computer.
>> All we know is that it will be consistently use less nodes on all computers
>>than the competing algorithm.  But it could also consistently take 20% more time
>>on all computers.  Of course we suspect better NNS performance will result in
>>better real world application performance, but this is BY NO MEANS CERTAIN.
>>TTS *demonstrates* that the thing really works, on at least 1 platform.
>>NNS suggests that the thing might work, on at least 1 platform.
>All true, but the "why" still remains.

Not sure exactly what you mean here, but NNS doesn't really explaing why
anything works.  Its just a way if indicating if it is probably going to work.



This page took 0.07 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.