Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test data

Author: Bob Durrett

Date: 13:14:10 09/12/03

Go up one level in this thread


On September 12, 2003 at 15:54:40, Gian-Carlo Pascutto wrote:

>On September 12, 2003 at 15:42:54, Bob Durrett wrote:
>
>>I really believe that the "tests" I'm running are quite revealing and far more
>>relevant to my interests.  I see no need to resort to counting wins and losses.
>>That would be a waste of time for me since I don't play against chess engines.
>>Why should I care whether or not one engine can whip another???????????
>
>It was *YOU* that stated:
>
>------------------
>In such a competition, I suspect that Crafty would come out on top of the rating
>list simply because Hyatt says Crafty does very little pruning.  [This assumes
>that the Crafty operator does not shut the engine off due to boredom.  : )]
>-------------------
>
>If you can't validly substantiate your claims and aren't willing to defend
>them, THEN PLEASE DON'T MAKE ANY.

Well, in the first place, I didn't make any claim except to say that I
"suspected" whatever.  That's a far cry from what you are talking about.  It
remains true that I "suspect" what I said I "suspected."  Surely you cannot
challenge that!  Do you never "suspect"?
>
>And as for your 'relevant' tests, are those the ones where you look at the
>score of multiple engines and conclude that the one that is 'out of line'
>must necessarily have 'something wrong' with it?

No, I never said "must necessarily have 'something wrong' with it."  If you
review the thread, you will see that those were your words, not mine.

Lets put this on a more serious note:  I claim that the results I'm seeing cause
me to tentatively conclude that Shredder needs more work insofar as it's
displayed position evaluation scores [and evaluation symbols] are concerned.

Do you know of any evidence to the contrary?  I have already said that I have
been allowing the engines to reach a minimum depth of at least 16, whatever that
is, and that I have been allowing all five engines to evaluate the same
positions.  Ten games times 40 moves per game is 400 moves.  I also do the same
in evaluating positions which occur in evaluation lines.  My guess is that I
have looked at over 1000 evaluations [depth > 15] produced by these engines
since I acquired them.  This is an ongoing process.  I will continue doing that
over the coming years!  Only GOD knows how many positions I will look at.

The bottom line:  I wish to know how to make the MOST sense out of the items of
information that the engines are giving me, in the context of serious
post-mortem game analysis.  Comparison of the engine outputs has been very
revealing!  [They are all different.  They would make a terrible Choir!]

Bob

>
>...Riiiiight
>
>--
>GCP



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.