Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Interpreting results from tests suites. Advice needed.

Author: Gian-Carlo Pascutto

Date: 12:02:27 06/08/02

Go up one level in this thread


On June 08, 2002 at 14:51:17, Gian-Carlo Pascutto wrote:

>On June 08, 2002 at 12:54:55, José Carlos wrote:
>
>>  So what's your experience in this matter? How would you interpret the results?
>
>Two observations:
>
>1) The new version solves less positions
>2) The new version does a bit better early on, but it loses out as time gets
>longer.
>
>If this were results of Sjeng, I'd drop the new version.
>
>No guarantee this is the best course of action, of course.

I have to add that in Sjeng, I measure nodes for each depth and average depth
reached as well. If a new version does noticeably better there, and is very
close in number of positions solved, it warrants further investigation.

Note that the testsuite is constructed in such a way that if you go a bit
faster, you generally also solve one or two more positions.

If despite going faster because of the pruning, you're still not solving
more positions, chances are good it makes the engine tactically weaker.

--
GCP



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.