Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Testing evaluation quality with test suits

Author: Uri Blass

Date: 07:39:15 12/14/03

Go up one level in this thread


On December 14, 2003 at 10:25:12, Albert Bertilsson wrote:

>Hi!
>
>I've started to look at the quality of the evaluation in Sharper and have some
>questions regarding a test suite I've found.
>
>The test suite is called QuietTest and has 81660 positions, from the name and
>the number of positions I guessed that this test should test the q-search
>function. So I tested it with q-search only and Sharper "solves" about 13% of
>the positions.
>
>My questions are:
>1. Is the test supposed to be run like I run it?

No

I do not think that the test is supposed to test the q-search function and the
name only say that the positions are quiet and not positions when there is
tactical line to find.


>2. What does your engine score in the test?

Never tested it and I doubt if the test is important.

I am not sure if the solutions are correct.
Based on my memory the way that it was generated is by having only positions
that some top engines agree about the same move after long time but it is
possible that they agree about a move that leads to a draw when there is an
alternative b that leads to draw.

I do not think that it was tested that all engines agree that the suggested move
get significantly higher positional score than other moves (something like at
least 0.1 pawns better).

Uri



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.