Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Bruce Moreland

Date: 23:49:17 12/18/02

Go up one level in this thread

On December 19, 2002 at 01:11:09, Omid David Tabibi wrote:

>On December 18, 2002 at 19:04:28, Bruce Moreland wrote:
>>On December 18, 2002 at 16:13:59, Omid David Tabibi wrote:
>>>I conducted self-play matches between std R=2 and std R=3. The results showed
>>>that std R=2 is superior, and that was enough for me.
>>If you are throwing other test suite data away because it contradicts your
>>conclusion, you should not be drawing conclusions from other test suite data.
>>You have test suite data supporting R=3 over R=2.
>>You have test suite data supporting VR=3 over R=2.
>>You have game play data supporting VR=3 over R=2.
>>You have unpublished game data supporting VR=3 over R=3.
>>You have to throw something out, so you are willing to throw out the test suite
>>data for R=3 over R=2 because it doesn't support your conclusion.
>>It is unclear why you choose to throw out this evidence rather than some other
>>evidence.  What leads you to believe that this evidence is spurious while the
>>other is not?
>In the previous version of the article, published as a Technical Report
>(, I didn't
>include Table 4, node counts on WCS positions. I didn't add it because I used
>WCS mainly for tactical comparisons, and thus thought that presenting the node
>counts would be superfluous.
>Now, did I want to hide that data? Does it make the evidence spurious?
>(Eventually, one of the reviewers of the paper suggested that I include that
>table anyway for the sake of completeness.)

I think that you misunderstand my point.

When I talk about throwing data out, I am saying that given the data you've
presented, you have to decide to ignore some of it, since it supports
contradictory conclusions.

The test suite data supports the superiority of R=3 over R=2.  The game play
data that you did not publish supports R=2 over R=3.

It can't be true that both are better, so you have to choose to ignore some of
your data, so that the remaining data can prove one of these conclusions.


This page took 0.02 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.