Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: inaccuracy of numerical score of (top) engines

Author: David H. McClain

Date: 08:54:52 10/30/03

Go up one level in this thread


On October 30, 2003 at 11:31:09, jefkaan wrote:

>just being curious, i wonder what sort of opinions
>people in this forum have about:
>1)  the (inc)accuracy of the scores of different engines
>in middle/ (practical)endgame positions, and 2):
>the variation the scores of different (top)engines
>(besides scaling factors of a positional nature).
>
>Reasons for my question:
>1) when playing standard with a reasonable engine
>(yace) against top-engines i'm setting my
>resign score lower every time, eg. when its
>below 3 i usually already can consider resigning;
>on the other hand, when i'm above +2, chances
>my side/engine will be winning are above 90 %
>(in lightning it can change a bit more,
>especially in gambit type of play, but
>then its just a matter of doubling the
>win/lose score i guess).
>2) when evaluating openings i wonder what
>the best positional engine would be, maybe
>Hiarcs9, maybe Shredder, i just dont know.
>Altogether they are not so much different
>in their judgments (ie numerical evaluations), i
>suspect, may on average about 0.3 pawn standard
>deviation difference in the cross correlations,
>or whatever you could use in statistical sense
>(although  in some difficult positions
>computers still sometimes are considerably
>different ie more stupid than human IM/GM evaluators,
>yep i know, or at least i believe that, at least
>when its a GM above the 2800 scale or so)
>
>Thanks in advance for your thoughts on this
>best regards,
>jefk
>PS and dont tell me btw i'm an idiot who
>doesnt know anything about chess, i know
>that as i've heard that already for some 10
>yrs now (mainly from a limited nr of people),
>but i'm still learning, you know :)

Unless you are personally a strong chess player, play the game out for a while
longer.  There is what is called a "Horizon effect" in that the chess engines is
simply counting "beans" and do not always see the tactical advantage or
disadvantage accurately at all stages of a game.

I have personally played several games where I was ahead as much as + 3.08 and
could do no better than a draw.  A GM would have seen it but I cannot.  I have
also played quite a few games where a disadvantage of perhaps - 1.75 by
evaluation turned into a win or a draw.  Some chess engines are notorious for
inaccurate evaluations (Shredder in particular).  I am not a strong player so I
cannot determine if my evaluation is really a win, draw or loss until sometime
later in the game.  You will know when this happens from experience.  Depending
on what engines your are using, evaluations are only a guide until the point of
no return has been reached and normally an engine's evaluation will catch up
with reality.

DHM



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.