Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Proving something is better

Author: Omid David Tabibi

Date: 22:11:09 12/18/02

Go up one level in this thread


On December 18, 2002 at 19:04:28, Bruce Moreland wrote:

>On December 18, 2002 at 16:13:59, Omid David Tabibi wrote:
>
>>I conducted self-play matches between std R=2 and std R=3. The results showed
>>that std R=2 is superior, and that was enough for me.
>
>If you are throwing other test suite data away because it contradicts your
>conclusion, you should not be drawing conclusions from other test suite data.
>
>You have test suite data supporting R=3 over R=2.
>
>You have test suite data supporting VR=3 over R=2.
>
>You have game play data supporting VR=3 over R=2.
>
>You have unpublished game data supporting VR=3 over R=3.
>
>You have to throw something out, so you are willing to throw out the test suite
>data for R=3 over R=2 because it doesn't support your conclusion.
>
>It is unclear why you choose to throw out this evidence rather than some other
>evidence.  What leads you to believe that this evidence is spurious while the
>other is not?
>

In the previous version of the article, published as a Technical Report
(ftp://ftp.cfar.umd.edu/TRs/CVL-Reports-2002/TR4406-tabibi.ps.gz), I didn't
include Table 4, node counts on WCS positions. I didn't add it because I used
WCS mainly for tactical comparisons, and thus thought that presenting the node
counts would be superfluous.

Now, did I want to hide that data? Does it make the evidence spurious?

(Eventually, one of the reviewers of the paper suggested that I include that
table anyway for the sake of completeness.)


>And if the test suite evidence is spurious, why include it in order to support
>VR=3 over R=2?  Throwing data out when it contradicts your conclusion and
>keeping it when it supports your conclusion doesn't seem proper.
>
>The autoplay games aren't up on the website, by the way.
>
>bruce



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.