Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Proving something is better

Author: Bruce Moreland

Date: 20:24:08 12/18/02

Go up one level in this thread


On December 18, 2002 at 22:43:45, Robert Hyatt wrote:

>On December 18, 2002 at 17:18:56, Bruce Moreland wrote:

>>But this doesn't address the problem I pointed out with Omid's thing:
>>
>>A takes 30 seconds, produces 50 answers.
>>
>>B takes 40 seconds, produces 55 answers.
>>
>>It makes absolutely no sense to say that B is better than A, and if the time
>>differential is large enough, and the difference in number of solutions is small
>>enough, it may make sense to say the reverse.
>
>
>I don't disagree there...

I've lost Omid, so I'll try to get you to see what I'm talking about then.

Omid proves that with his program, VR=3 is better than R=2.

He assumed before he started that R=2 is better than R=3.  So he never tested
R=3 to create a baseline, before tweaking R=3 to create VR=3.

But he does include numbers for R=3 for a couple of test suites, and the
solution numbers are almost identical with R=2.  Of course R=3 takes about 40%
as much time as R=2, so these numbers are superior.

So for these suites, R=3 is better than R=2 for this program.  How do you
compare a variant of R=3 against anything when your program is already acting as
if R=3 is better than R=2?  The numbers are either bogus or point to a larger
conclusion.

bruce



This page took 0.15 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.