Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: To evaluate/test 'positional skills' of chess engines

Author: Uri Blass

Date: 13:20:20 12/07/02

Go up one level in this thread


On December 07, 2002 at 14:09:48, Uri Blass wrote:

>On December 07, 2002 at 11:36:03, Ingo Lindam wrote:
>
>>On December 07, 2002 at 11:25:03, Uri Blass wrote:
>>
>>>On December 07, 2002 at 05:04:33, Ingo Lindam wrote:
>>>
>>>>Hello all,
>>>>
>>>>concerning the current discussion about testing positional skills
>>>>I like to add the following:
>>>>
>>>>I think to evaluate/test positional skills of an chess engine you don't need to
>>>>have positions without tactics (what I think is impossible, because for
>>>>everything true in a chess position there is a tactical proof if you are just
>>>>able to look deep enough into the tree).
>>>>
>>>>We agree that also in a nearly 'tactical solution free' (within a certain scope)
>>>>position with a one and only best move it could happen that the engine plays the
>>>>right move for the wrong reason.
>>>>
>>>>So why not test the other direction? Assume that you have a true evaluation for
>>>>a position. This true evaluation doesn't need to be exact it can also be: "White
>>>>is atleast equal (or better)." So as long as the engine claims a clear advantage
>>>>for black, it fails the test. If the score corresponds with your own evaluation
>>>>you are ofcourse not able to judge about the engine as long as you not checked
>>>>the reason, why it gives this evaluation. Ofcourse your test positions prefers
>>>>the engine to give the wrong evaluation instantly because of e.g. being a pawn
>>>>up. The deeper the engine goes into the search tree keeps to give the wrong
>>>>evaluation the worse is the positional evaluation of the maschine (at least at
>>>>this special kind of positions). And because you know at least one reason, why
>>>>the position shouldn't be judge that worse as the engine does, you should also
>>>>be able to name some/a positional feature/s the engine doesn't evaluate right.
>>>>
>>>>Giving an example:
>>>>
>>>>[D] r3kb1r/1p1q1ppp/p1n1p3/3pN1B1/3Pb3/1Q2P3/PP2BPPP/2R1K2R b Kkq - 0 1
>>>>
>>>>position after 12.Se5
>>>>
>>>>Analysis by Fritz 7:
>>>>(sorry for the german letters: S=N,L=B,D=Q,T=R and K=K)
>>>>
>>>>12...Sxe5 13.f3 Sg6 14.fxe4 h6 15.Lf4 Sxf4 16.exf4 dxe4
>>>>³ (-0.69) Tiefe: 8/21 00:00:00 151kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.f3 h6 16.Lh4 Lf5 17.Kf1
>>>>³ (-0.69) Tiefe: 9/24 00:00:00 400kN
>>>>12...Sxe5 13.f3 Sg6 14.fxe4 h6 15.Lf4 Sxf4 16.exf4 dxe4 17.De3 Lb4+
>>>>µ (-0.78) Tiefe: 10/27 00:00:01 935kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.f3 h6 16.Lf4 Lf5 17.Kf1 Tc8 18.Kg2
>>>>µ (-0.75) Tiefe: 11/29 00:00:04 2493kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.f3 h6 16.Lf4 Lg6 17.Kf1 Le7 18.Ld3 Lxd3+
>>>>19.Dxd3
>>>>µ (-0.78) Tiefe: 12/31 00:00:10 5134kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.f3 h6 16.Lf4 Lf5 17.Kf1 Tc8 18.Txc8+ Dxc8
>>>>µ (-0.84) Tiefe: 13/36 00:00:28 15940kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.f3 h6 16.Lf4 Lf5 17.Kf1 Tc8 18.Txc8+ Dxc8
>>>>µ (-0.78) Tiefe: 14/38 00:01:06 39095kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.f3 Lf5 16.Kf2 b5 17.Lf4 Tc8 18.Txc8+ Dxc8
>>>>µ (-0.84) Tiefe: 15/42 00:02:47 102460kN
>>>>12...Sxe5 13.dxe5 Lxg2 14.Tg1 Le4 15.Db6 Tc8 16.Txc8+ Dxc8 17.Kd2 d4 18.Dxd4
>>>>Dc2+
>>>>µ (-0.72) Tiefe: 16/46 00:08:08 293734kN
>>>>
>>>>Still at ply 16 the machine evaluates (in my opinion wrongly) an advantage for
>>>>black, although the PV contains th ebest moves for white in the beginning.
>>>>(going one move back in the position you will see that the engine will not play
>>>>Ne5 within 17 plys).
>>>>
>>>>Giving the engine some more (white) moves towards 16. Kd2 the evaluation will
>>>>change into the right direction.
>>>>
>>>>in the game it follows:
>>>>
>>>>12... Nxe5 13. dxe5 Bxg2 14. Rg1
>>>>Be4 15. Qb6 Rc8
>>>
>>>
>>>The question is if 12.Ne5 was the only best move.
>>>I doubt it
>>>
>>>Movei prefers 15...h6
>>>
>>>Is there a win for white after  15...h6?
>>>
>>>A possible line is 16.Rc7 Qa4 17.a3 hxg5 18.Qxb7 Bb4+ 19.Kf1 0-0 20.Qxb4 Qxb4
>>>21.axb4 and the endgame seems to be equal.
>>>
>>>Uri
>>
>>My question was not whether Ne5 is doubtless the one and only best move. I am
>>sure...that black is not better here, after white plays Ne5 and nevertheless the
>>engine (Fritz 8) gives a clear advantage to black. And (if I am right) that
>>should be a sign that the positional evaluation of the engine is not optimal is
>>this position here, shouldn't it?
>
>I am not so sure about it.
>
>The moves of Fritz in the pv may not be the best moves for black(I suspect that
>13...Bxg2 is a mistake).
>
>
>I guess that the fact that 15.Qb6 can force a draw is a tactical observation
>that Fritz is not able to see.
>
>I checked with movei and here is analysis 2 plies later
>
>Movei is unable to see the problem at depth 12 and wants to take g2.
>The problem is lack of important extensions
>Bxg2 should trigger an extension because Rg1 is forced but movei does not know
>it.
>Movei can see at depth 13 that Bxg2 Rg1 Be4 is not good and the pv is changed to
>Bxg2 Rg1 h6
>
>You can see by the score that it is tactics(the score dropped from 0.89 to
>0.13).
>
>Movei fails high on h6 later
>
>
>depth=12 +0.89 e4g2 h1g1 g2e4 f2f3 e4f5 b3b6 a8c8 e1d2 c8c1 g1c1 h7h6 c1c7 f8b4
>d2c1
>Nodes: 17303621 NPS: 161068
>Time: 00:01:47.43
>
>depth=13 +0.13 e4g2 h1g1 h7h6 g5h6 h8h6 g1g2 a8c8 c1d1 d7c7 b3a4 e8d8 a4f4 h6h3
>f4g5 d8e8 e1f1
>Nodes: 47304313 NPS: 159569
>Time: 00:04:56.45
>depth=13 +0.14 h7h6
>Nodes: 69727812 NPS: 160422
>Time: 00:07:14.65
>depth=13 +0.43 h7h6
>Nodes: 77614594 NPS: 159585
>Time: 00:08:06.35
>depth=13 +0.58 h7h6 f2f3 e4f3 e2f3 h6g5 e3e4 d5d4 f3e2 f8e7 b3b6 d4d3 c1d1 e7d8
>b6b4
>Nodes: 95817407 NPS: 158874
>Time: 00:10:03.10
>
>Uri


It seems that Bxg2 is simply a tactical mistake also based on yace only material
and it seems that after h6 black has the advantage.


New position
[D]r3kb1r/1p1q1ppp/p3p3/3pP1B1/4b3/1Q2P3/PP2BPPP/2R1K2R b Kkq - 0 1

Analysis by Yace 0.99.56(only material):

2...Be4xg2
  µ  (-0.80)   Depth: 1   00:00:02
2...Be4xg2 3.Rh1-g1
  µ  (-0.80)   Depth: 2   00:00:02
2...Be4xg2 3.Rh1-g1 Bg2-e4
  µ  (-0.80)   Depth: 3   00:00:02
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4
  µ  (-0.80)   Depth: 4   00:00:02
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5
  µ  (-0.80)   Depth: 5   00:00:02  5kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1
  µ  (-0.80)   Depth: 6   00:00:02  21kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1 g7-g6
  µ  (-0.80)   Depth: 7   00:00:02  68kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1 g7-g6 6.Ke1-f1
  µ  (-0.80)   Depth: 8   00:00:02  171kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1 g7-g6 6.Ke1-f1 Be4-f5
  µ  (-0.80)   Depth: 9   00:00:03  443kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1 g7-g6 6.Ke1-f1 Be4-f5
7.Bh4-g3
  µ  (-0.80)   Depth: 10   00:00:05  1318kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1 g7-g6 6.Ke1-f1 Be4-f5
7.Bh4-g3 Ke8-e7
  µ  (-0.80)   Depth: 11   00:00:09  3017kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Bg5-h4 b7-b5 5.Qb3-d1 g7-g6 6.Ke1-f1 Be4-f5
7.Qd1-d2 Bf5-h3+ 8.Kf1-e1 Rh8-g8
  µ  (-0.80)   Depth: 12   00:00:19  8520kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Qb3-b6 Ra8-c8 5.Rc1xc8+ Qd7xc8 6.Ke1-d2 Be4-f5
7.Rg1-c1 Qc8-d7 8.Be2xa6 b7xa6 9.Qb6-b8+ Qd7-c8 10.Rc1xc8+ Ke8-d7
  ³  (-0.40)   Depth: 13   00:00:53  26420kN
2...Be4xg2 3.Rh1-g1 Bg2-e4 4.Qb3-b6 Ra8-c8 5.Rc1xc8+ Qd7xc8 6.Ke1-d2 Bf8-e7
7.Bg5xe7 Ke8xe7 8.Rg1xg7 Ke7-e8 9.Rg7-g5 Be4-f5 10.Be2-d1
  =  (0.00)   Depth: 13   00:01:12  35615kN
2...h7-h6 3.Bg5-h4 Be4xg2 4.Rh1-g1 Bg2-e4 5.Rg1-g2
  =  (-0.01)   Depth: 13   00:02:57  90360kN
2...h7-h6 3.Bg5-h4 Be4xg2 4.Rh1-g1 Bg2-e4 5.Ke1-f1 b7-b5 6.Bh4-g3 Ra8-c8
7.Rc1xc8+ Qd7xc8 8.Be2-d1 Ke8-d8 9.f2-f3 Be4-f5
  µ  (-0.80)   Depth: 13   00:03:29  106018kN
2...h7-h6 3.Bg5-h4 Be4xg2 4.Rh1-g1 Bg2-e4 5.Ke1-f1 b7-b5 6.Bh4-g3 Ra8-c8
7.Rc1xc8+ Qd7xc8 8.Be2-d1 Ke8-d8 9.f2-f3 Be4-f5 10.Rg1-g2
  µ  (-0.80)   Depth: 14   00:06:10  187313kN
2...h7-h6 3.Bg5-h4 Be4xg2 4.Rh1-g1 Bg2-e4 5.Ke1-f1 b7-b5 6.Bh4-g3 Ra8-c8
7.Rc1xc8+ Qd7xc8 8.Be2-d1 Ke8-d8 9.f2-f3 Be4-f5 10.Rg1-g2 g7-g6
  µ  (-0.80)   Depth: 15   00:14:12  435953kN

(Blass, Tel-aviv 07.12.2002)


Uri



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.