Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Very interesting position.. Crafty vs ProDeo

Author: Vasik Rajlich

Date: 02:10:27 08/25/04

Go up one level in this thread


On August 24, 2004 at 14:18:52, Álvaro Begué wrote:

>On August 24, 2004 at 13:24:03, Vasik Rajlich wrote:
>
>>On August 24, 2004 at 11:29:31, Anthony Cozzie wrote:
>>
>>>On August 24, 2004 at 10:37:02, Vasik Rajlich wrote:
>>>
>>>>On August 24, 2004 at 04:57:02, Tord Romstad wrote:
>>>>
>>>>>On August 23, 2004 at 23:04:13, Robert Hyatt wrote:
>>>>>
>>>>>>This particular case is not a big issue with me.  I personally believe that the
>>>>>>+2 eval is wrong.  And it would be interesting to keep the same position, but
>>>>>>move a white piece or pawn and see what happens and if black would still play
>>>>>>the same.  IE maybe white bishop at d1 rather than a1.  That changes the
>>>>>>position although I have not given a lot of thought to this...
>>>>>
>>>>>I think this particular case *should* be a big issue to you, and I'll try
>>>>>to explain why.  The kind of position we have after 1... Nxh3 2. Kxh3
>>>>>Rh6+ 3. Kg4 is extremely hard to evaluate accurately.  It is very possible
>>>>>that your statement that +2 for black is wrong.  Clearly, it is very risky
>>>>>to evaluate such positions as winning for black, and doing so will sometimes
>>>>>cause your engine to lose games.  On the other hand, it is just as risky
>>>>>to evaluate the position as winning for white.  With the white king so
>>>>>exposed and no easy way home, it is very possible that black has a
>>>>>winning attack.
>>>>>
>>>>>Not only is the position very difficult to evaluate, it is also a very
>>>>>important position.  The line is almost forced, and the likelihood that
>>>>>the resulting position is won for one of the players is very high.
>>>>>
>>>>>The best way I have found to handle such positions is to extend.  When
>>>>>one side has a winning material advantage, but the other side has a
>>>>>very dangerous attack, extend by half a ply or so.  This will often
>>>>>help you to discover and correctly evaluate sacrifical kingside attacks
>>>>>several plies earlier, and the cost is very low in most positions
>>>>>(because such attacks are rather rare).
>>>>>
>>>>>Making the static eval aware of its limitations offers many interesting
>>>>>possibilities, and I think there are many valuable and important ideas
>>>>>waiting to be found by the adventurous programmer here.  The basic
>>>>>idea is to extend in positions where the static eval is likely to be
>>>>>highly inaccurate, and to reduce in positions where it is likely to
>>>>>be very accurate (internal node recognizers is an extreme special case).
>>>>>
>>>>>Tord
>>>>
>>>>You're (probably) right that these are good positions in which to extend, but
>>>>IMO the justification should be put differently.
>>>>
>>>>There should be a 1-to-1 mapping between the output of your static eval and
>>>>winning %. (As I am sure you agree.) If you're uncertain - it just means that
>>>>the score needs to be brought closer to 0. So, having eval return {eval,
>>>>uncertainty} is redundant. However, what eval can return is {eval,
>>>>"likelyhood-that-one-extra-ply-of-search-will-change-the-eval"}. This second
>>>>term should be higher when the king is exposed, or when there are passed pawns,
>>>>and it should be the justification for this sort of extending.
>>>>
>>>>Hope this is useful (rather than a nitpick) ..
>>>>
>>>>Vas
>>>
>>>I think Tord's point is that big kingsafety scores are _always_ inaccurate ;)
>>>
>>>anthony
>>
>>If that's the case, then evals for positions with big kingside scores should
>>always be adjusted in the direction of 0.
>>
>>Some time ago I considered returning an evaluation uncertainty from my eval. I
>>decided that no such thing existed - it's an inherent part of the eval score.
>>
>>Vas
>
>A quantitative way of doing this would be to estimate a mean and a variance of
>the score (say, trying to predict with the evaluation function what the score of
>a depth 8 search would return). Let's say we have a function
>sigma(x):=1/(1+exp(-x)) that returns our best estimate of the number of points
>you are going to score in the game given that the evaluation is x. Then you can
>use your estimate in the variance of the evaluation by computing the expected
>value E(sigma(N(mean,variance))), where N is a normal distribution. The effect
>is actually moving the evaluation of uncertain positions towards zero.
>
>This can be very desirable, as the program will try a crazy attack when it's
>losing but it will try to cash in for material instead of keeping an uncertain
>attack when it's winning.
>
>In any case, this complexity is encapsulated in the evaluation function and
>there is no need to return more than one number, as Vas said.

Yes, that is exactly what I meant. Internally your evaluation function could
keep a "raw score" and a variance, which is converted to a single value only at
the end. If you believe that evaluating attacks is a less certain activity than
evaluating material (I am not 100% sure I agree with this actually), then
attacks would be given a greater variance, and the net effect is that they would
weigh more in bringing you toward a score of 0 (ie compensating for lost
material) than in taking you away from 0.

What the static eval function can return, in addition to its static score, is an
estimate of how much benefit could be derived from additional search. In more
"volatile" positions, this benefit should be greater - and this is a way to
capture the "Tord extension" for unclear positions.

Anyway - this is all just a theoretical framework. The tricky thing is to fill
in the gaps.

Vas



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.