Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: WM Test Position 1 - Good Position or Proving Weakness of the Test!

Author: Manfred Meiler

Date: 13:58:04 06/11/04

Go up one level in this thread


On June 11, 2004 at 14:55:38, Rolf Tueschen wrote:

>On June 11, 2004 at 13:15:25, Ed Schröder wrote:
>
>>On June 11, 2004 at 07:29:20, Rolf Tueschen wrote:
>>
>>>On June 11, 2004 at 02:14:32, Ed Schröder wrote:
>>>
>>>>On June 09, 2004 at 10:13:30, Franz Hagra wrote:
>>>>
>>>>>Te3 is not winning at all - its a draw (only in the original game black wins).
>>>>>
>>>>>Te3 is the key to draw the position, but its not essential to play it as first
>>>>>move at all - so the test position is clear in logical human sence, but not
>>>>>under test conditions, because the test only works correct, when only Te3 as
>>>>>first move is found!
>>>>>
>>>>>Tad8 also leads to a draw position like Te3 - so the TEST POSITION is not
>>>>>correct at all.
>>>>
>>>>[d]r3r1k1/1pq2pp1/2p2n2/1PNn4/2QN2b1/6P1/3RPP2/2R3KB b - -
>>>>
>>>>1..Re3 is a sound positional attacking move and according to my own brainchild
>>>>there is a difference of 0.25 in score between 1..Re3 and 1..Rad8. The position
>>>>IMO is a fine one to test the strategic insight of a chess program.
>>>>
>>>>My best,
>>>>
>>>>Ed


>>>
>>>Ed,
>>>
>>>you did NOT comment on the main finding Hagra has published here in
>>>http://www.talkchess.com/forums/1/message.html?369557!
>>>
>>>I translate a second time into English:
>>>
>>>a) machine FRITZ 8 on AMD 1400 gets a solution time of 1 sec and that means
>>>highest points for position no. 1 (which you gave thankfully above)
>>>
>>>b) machine FRITZ 8 on AMD 2800 gets a solution time of 480 sec!! So that it gets
>>>way worse points in position no. 1!!
>>>
>>>Here is my verbal explanation (all found by Hagra):
>>>
>>>a stronger [!] machine on better hardware (do you accept that or do you claim
>>>that AMD 1400 is STRONGER than AMD 2800?) is able to make a deeper [!!]
>>>calculation and therefore finds the variation with first Rad8 - NOT as a final
>>>solution, Ed! But as a variation, before it THEN comes back to Re3. Now, the
>>>point is that such a behaviour is by far a sign for weaker strength but for
>>>_better_ strength. But alas, Ed, the so called "WM-Test" of Dr. Mikhail Gurevich
>>>gives to the weaker machine more points than for the stronger machine.


>>
>>This is common problem with positions which nature is positional. Take the start
>>position for example, the moves 1.e4, 1.d4, 1.c4 and 1.Nf3 will produce scores
>>that are very close to each other, in this case 4 moves with almost identical
>>scores. What you see is that chess engines tend to switch from 1.e4 to 1.d4
>>frequently and that the speed of the PC actually introduces a random element as
>>in this case with Fritz.


>
>
>This is still not what is happening in the case of Fritz on different hardware
>and WM-Test position number 1.
>
>a) at first Fritz 8 on AMD 2800 goes directly for Re3
>
>b) then at a depth of 12 it changes to Rad8
>
>c) later it comes back to Re3 (therefore it gets 480 sec as result)



First of all - sorry for my bad english.
I'm the one who made the tests of (in the meantime) 260 chess engines (including
Fritz 8.0.0.23) in the test suite "WM-Test" since around 3 years on my (older)
PC AMD Athlon Thunderbird 1400 MHz.
My detailed results (in Excel) of 230 tested engines are available for free
download at http://www.computerschach.de/test/index.htm (including the 100
commented test positions of "WM-Test" in various formats cbh/cbf/pgn/epd).

I do not have Hagra's AMD Athlon XP 2800+ (AFAIK this machine offers real 2083
MHz) but an Intel Pentium 4 with 3.0 GHz (HT) which should give about the same
power for chess analysis as Hagra's AMD XP 2800+.
So I tried the test position 1 (...Re3, see above) of "WM-Test" with the
mentioned Fritz engine 8.0.0.23 (Fritz 8.exe of 2003 March 3rd) with 256 MB Hash
in analyse mode (AFAIK Hagra used the automatic feature "Process test set" of
Fritz) once again - this time on my P4-3.0; and here are the analysis of Fritz
8.0.0.23 on P4-3.0:

  1...Db6!
² (0.66) Tiefe: 6/24 00:00:00 401kN
1...Db6 2.bxc6 bxc6 3.Tcd1 Db4 4.Lxd5 Sxd5 5.Dxb4 Sxb4 6.Kf1
² (0.66) Tiefe: 7/24 00:00:00 564kN
1...Te3!
² (0.62) Tiefe: 7/26 00:00:01 926kN
1...Te3 2.Lg2 Tea3 3.e3 Da5 4.Td3 cxb5 5.Sxb5
² (0.59) Tiefe: 7/26 00:00:01 1007kN
1...Te3 2.Sd3 Txg3+ 3.Lg2 Txg2+ 4.Kxg2 Se4 5.Tdd1 Sf4+ 6.Kf1
² (0.66) Tiefe: 8/29 00:00:02 1968kN
1...Te3 2.Lg2 Tea3 3.Kh2 Sb6 4.Dc2 Sbd5 5.bxc6 bxc6 6.Tcd1
² (0.56) Tiefe: 9/28 00:00:05 5733kN
1...Te3 2.Lg2 Tea3 3.Scb3 T3a4 4.Dc5 Sb4 5.Dg5 Ld7 6.Sc5 cxb5
² (0.59) Tiefe: 10/31 00:00:14 15330kN
1...Te3 2.Lg2 Tea3 3.Scb3 T8a4 4.Dc5 Sb4 5.bxc6 b6 6.Dg5 Ta2 7.Txa2
² (0.66) Tiefe: 11/33 00:00:34 37372kN
1...Te3 2.Lg2 Txg3 3.e3 cxb5 4.Dxb5 b6 5.Scb3 Txg2+ 6.Kxg2 De5 7.Th1 De4+ 8.Kg1
² (0.69) Tiefe: 12/36 00:01:29 100326kN
1...Te3 2.Lg2 Lh3 3.bxc6 bxc6 4.Lf3 Tea3 5.Scb3 T8a4 6.Dxc6 Dxc6 7.Txc6 Ld7
² (0.66) Tiefe: 13/36 00:04:53 335752kN
1...Te3 2.Lg2 Txg3 3.e3 Txg2+ 4.Kxg2 cxb5 5.Dxb5 b6 6.Sd3
² (0.56) Tiefe: 14/39 00:15:08 1035797kN
1...Te3 2.Lg2 Txg3 3.e3 Txg2+ 4.Kxg2 De5 5.Ta2 Txa2 6.Dxa2 b6 7.Da8+
² (0.37) Tiefe: 15/44 00:45:49 3135852kN

As you can see (also at http://f23.parsimony.net/forum50826/messages/100856.htm)
Fritz 8.0.0.23 found the solution ...Re3 (Te3) à tempo and hold it over more
than 1 hour (with reached depth 16/16 after 1,5 hours) - without any changes to
...Rad8 (or to another move) in the meantime.
All this happened under exact the identical test conditions which I use since 3
years for my testings of now 260 engines in "WM-Test" (aside from the here used
machine P4-3.0 instead of the usual AMD 1400).



>
>However the same engine on weaker hardware AMD 1400 does never change to Rad8
>because it can't go deep as 12. Therefore here Fritz 8 gets the result 1 sec
>which is the best possible. For this apparent contradiction you must find a
>reasonable solution in your test suite.



A reasonable solution for this "apparent contradiction" could be: Your claim is
simply wrong.
How do you know that "my weaker hardware AMD 1400 (does never change to Rad8
because it) can't go deep as 12"? Did you test it yourself?
The truth is: My AMD Athlon Thunderbird 1400 MHz (real 1400 MHz!) - so the
machine which I used for testing 260 engines (also Fritz 8.0.0.23) in "WM-Test"
- reached in this position ply depth 12 after 2:19, depth 13 after 7:36 and
depth 14 after 23:00 ... found Re3 (Te8-e3) immediately and sticked at this Re3
even after more than 20 minutes without any changes to another move(s) - please
look at the analyse lines of Fritz 8.0.0.23 on my AMD 1400:

1...Ta8-a3 2.e2-e3 Dc7-a5 3.Td2-d3 c6xb5 4.Sd4xb5
  ²  (0.62)   Tiefe: 5/24   00:00:00  145kN
1...Ta8-a3 2.e2-e3 Sd5-e7 3.Lh1-g2 b7-b6 4.Sc5-e4
  ²  (0.69)   Tiefe: 6/22   00:00:00  195kN
1...Dc7-b6!
  ²  (0.66)   Tiefe: 6/24   00:00:00  282kN
1...Dc7-b6 2.b5xc6 b7xc6 3.Tc1-d1 Db6-b4 4.Lh1xd5 Sf6xd5 5.Dc4xb4 Sd5xb4
6.Kg1-f1
  ²  (0.66)   Tiefe: 7/24   00:00:01  564kN
1...Te8-e3!
  ²  (0.62)   Tiefe: 7/26   00:00:01  926kN
1...Te8-e3 2.Lh1-g2 Te3-a3 3.e2-e3 Dc7-a5 4.Td2-d3 c6xb5 5.Sd4xb5
  ²  (0.59)   Tiefe: 7/26   00:00:01  1007kN
1...Te8-e3 2.Sc5-d3 Te3xg3+ 3.Lh1-g2 Tg3xg2+ 4.Kg1xg2 Sf6-e4 5.Td2-d1 Sd5-f4+
6.Kg2-f1
  ²  (0.66)   Tiefe: 8/29   00:00:05  1968kN
1...Te8-e3 2.Lh1-g2 Te3-a3 3.Kg1-h2 Sd5-b6 4.Dc4-c2 Sb6-d5 5.b5xc6 b7xc6
6.Tc1-d1
  ²  (0.56)   Tiefe: 9/28   00:00:12  5733kN
1...Te8-e3 2.Lh1-g2 Te3-a3 3.Sc5-b3 Ta3-a4 4.Dc4-c5 Sd5-b4 5.Dc5-g5 Lg4-d7
6.Sb3-c5 c6xb5
  ²  (0.59)   Tiefe: 10/31   00:00:26  15330kN
1...Te8-e3 2.Lh1-g2 Te3-a3 3.Sc5-b3 Ta8-a4 4.Dc4-c5 Sd5-b4 5.b5xc6 b7-b6
6.Dc5-g5 Ta3-a2 7.Td2xa2
  ²  (0.66)   Tiefe: 11/33   00:00:56  37372kN
1...Te8-e3 2.Lh1-g2 Te3xg3 3.e2-e3 c6xb5 4.Dc4xb5 b7-b6 5.Sc5-b3 Tg3xg2+
6.Kg1xg2 Dc7-e5 7.Tc1-h1 De5-e4+ 8.Kg2-g1
  ²  (0.69)   Tiefe: 12/36   00:02:19  100326kN
1...Te8-e3 2.Lh1-g2 Lg4-h3 3.b5xc6 b7xc6 4.Lg2-f3 Te3-a3 5.Sc5-b3 Ta8-a4
6.Dc4xc6 Dc7xc6 7.Tc1xc6 Lh3-d7
  ²  (0.66)   Tiefe: 13/36   00:07:36  335752kN
1...Te8-e3 2.Lh1-g2 Te3xg3 3.e2-e3 Tg3xg2+ 4.Kg1xg2 c6xb5 5.Dc4xb5 b7-b6
6.Sc5-d3
  ²  (0.56)   Tiefe: 14/39   00:23:00  1035797kN


... only to give some facts to this discussion from my side.

Manfred



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.