Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Do hash tables confuse Crafty's matefinding routines?

Author: Richard A. Fowell (fowell@netcom.com)

Date: 12:39:31 12/27/98

Go up one level in this thread


On December 27, 1998 at 10:34:40, Robert Hyatt wrote:

>On December 26, 1998 at 18:51:46, Richard A. Fowell (fowell@netcom.com) wrote:
>
>>Do hash tables confuse your matefinding routines?
>>
>>I tried this mate in 7:
>>8/krK5/p7/P7/6P1/8/7Q/8 w - -
>>
>>on two different chess programs, by two different authors,
>>and both reported erroneously short mates when their hash tables were on.
>>
>>With hash tables off, the correct solution was given by both programs.

<snip>

>
>This has to be a bug.  Hash tables can't find a "shorter mate" than actually
>exists on the board, unless there is a bug.  Hash tables can let a shallow
>search see "much deeper" than you would expect (position fine#70 is the classic
>example that programs solve at a much shallower depth than is actually
>possible).

A bug, certainly, but I'm seeing evidence of that bug in three programs,
and one of them is Crafty.

On r.g.c.c., a mate in 9 was posted:
kr6/2K5/p7/P7/6PQ/8/8/8 w - -

One Crafty report found the mate in 9,
but two evaluated it as mate in 8. Digging through DejaNews, I've found
and appended a few pertinent posts:

Post #1 - by Robert Hyatt

You responded to Ed's post that some Crafty users were reporting the mate
in 9, but that Ed's copy of 16.2 reported mate in 8. Ed further pointed
out that the PV given by Crafty could be improved to extend the defence
to mate in 9. It seems you wanted more evidence that it was truly a
mate in 9 before investigating ... you said:

"But before I say anything the above should be looked at carefully
 to be sure there isn't some other  unnoticed resource that keeps the
 mate correct..."

Post #2 - Heiner Marxen finds the two mates in 9 using his special-purpose
          mate-finding program.

Post #3 - Mike Byrne reports Crafty finding a mate in 8,
          with the other keymove (Qh1+) after
          some tweaking of Crafty parameters.

Post #4 - Heiner Marxen points out an improvement to Black's
          defence in the Qh1+ variation, and asks for confirmation
          of the mate in 8 (since, if it existed, there would be
          an undetected bug in Heiner's program)

Post #5 - Mike Byrne (four days later) says he added a Crafty mate-finding
          routine, and that "this is a mate in 9". I take this to mean
          that he confirmed that the mate in 8 was erroneous. He said
          he emailed you the matefinding code.

I ran this problem on Sigma Chess, and it said there was no mate in 8.

In summary, I think there's confirmation that Crafty's eval of Mat08
in both Ed Collin's and Mike Byrne's copy of Crafty was erroneous
(per  Heiner Marxen, Mike Byrne, and myself), and that Crafty shares this
bug of erroneously short mate evals. Based on my work with Sigma Chess, it
seems that hash tables handling is a likely culprit.

Since this seems to be a widespread issue, and it seems to affect Crafty,
I was hoping you could help track down what causes this bug, and warn people
about it (heck, as widespread as this seems to be, it might be worth an
(ICCA paper).

Regards,

Richard A. Fowell (fowell@netcom.com)

===============================================
Edward D. Collins <ecollins@inficad.com> wrote:
:>> kr6/2K5/p7/P7/6PQ/8/8/8 w - - 0 0

: Bernhard Bauer wrote...

:> Crafty16.2 on a 2xPPro 233MHz finds the
:> mate in 9 on ply 11.

:> The move 1. g5 is found in 6 sec and the
:> whole search is finished in 1:42.


:> Here some output from crafty:

:>       10->   9.08   9.92   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
:>                            Qh7+ Ka8 5. Qc7 Rb1 6. Qd8+ Rb8 7.
:>                            Qxb8+ Kxb8 8. g7 Ka7 9. Kd6
:>       11    10.84     ++   1. g5!!
:>       11     1:31  Mat09   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
:>                            Qh7+ Ka8 5. g7 Rc8+ 6. Kb6 Rb8+ 7.
:>                            Kxa6 Rb7 8. g8=Q+ Rb8 9. Qb7#
:>       11->   1:42  Mat09   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
:>                            Qh7+ Ka8 5. g7 Rc8+ 6. Kb6 Rb8+ 7.
:>                            Kxa6 Rb7 8. g8=Q+ Rb8 9. Qb7#
:>      time=1:42  cpu=200%  mat=5  n=30057337  fh=97%  nps=292585
:>      ext-> checks=6269540 recaps=290 pawns=277457 1rep=691838
:>      predicted=0  nodes=30057337  evals=126326
:>      endgame tablebase-> probes done=0  successful=0  maxd=32
:>      hashing-> trans/ref=55%  pawn=99%  used=w89% b99%
:>      SMP->  split=2340  stop=534  data=9/64  cpu=3:26  elap=1:42
:> mate in 9 moves.


: Interesting.  I hadn't downloaded 16.2 yet so after seeing your post I
: did so.  I setup the problem and then let it run while I took a shower
: and had breakfast.

: I was surprised to discover later that I received different results.  In
: my log file, Crafty reported a mate in 8!

Before you can compare results, all facets of the experiment have to be
the same.  You probably didn't use the same hash size, and you definitely
didn't use the parallel search, which brings its own set of quirks into
the game..

If anyone is interested, we can certainly discuss "parallel anomalies"
as they can be quite interesting...


:    10    30.49     ++   1. Qh1+!!
:    10->  32.55  10.27   1. Qh1+
:    11    32.77     ++   1. Qh1+!!
:    11    52.23  Mat10   1. Qh1+ Ka7 2. Qh2 Rb7+ 3. Kc8 Rb5
:                         4. Qc7+ Ka8 5. Qc6+ Ka7 6. Kc7 Rb8
:                         7. Qb6+ Rxb6 8. axb6+ Ka8 9. b7+ Ka7
:                         10. b8=Q#
:    11    42:47  Mat08   1. g5 Rb7+ 2. Kd6 Ka7 3. g6 Ka8 4.
:                         g7 Rxg7 5. Qh8+ Ka7 6. Qxg7+ Ka8 7.
:                         Kc6 Kb8 8. Qb7#
:    11->  43:35  Mat08   1. g5 Rb7+ 2. Kd6 Ka7 3. g6 Ka8 4.
:                         g7 Rxg7 5. Qh8+ Ka7 6. Qxg7+ Ka8 7.
:                         Kc6 Kb8 8. Qb7#
:    time=43:35  cpu=99%  mat=5  n=120830485  fh=98%  nps=46197
:    ext-> checks=26133140 recaps=653 pawns=1002111 1rep=2880703
:    predicted=0  nodes=120830485  evals=190387
:    endgame tablebase-> probes done=1751199  successful=104850  maxd=32
:    hashing-> trans/ref=47%  pawn=13%  used=w99% b99%


: The thing is, after 5.Qh8+, there is no mate in 8 as Crafty can delay
: the mate by one move by interposing with the Rook (5... Rg8 6.Qxg7 Kb7
: 7.Qf7+ Kb8 8.Kc6 Ka8 9.Qb7#)

: Not sure yet why this happened...

First Crafty isn't supposed to find the "shortest" mate, although it should
never find one shorter than really exists...  But before I say anything the
above should be looked at carefully to be sure there isn't some other
unnoticed resource that keeps the mate correct...



: Like I said, I enjoy watching computers work on this one.


: - Ed


--
Robert Hyatt                    Computer and Information Sciences
hyatt@cis.uab.edu               University of Alabama at Birmingham
(205) 934-2213                  115A Campbell Hall, UAB Station
(205) 934-5473 FAX              Birmingham, AL 35294-1170
======== end of post 1, start of post 2 ===========

Heiner  Marxen
Email:  heiner@DrB.Insel.DE
Date:   1998/12/22
Forums: rec.games.chess.computer

In article <75nppn$qj6$1@news.inficad.com>,
Edward D. Collins <ecollins@inficad.com> wrote:
>Here's a position I still have fun watching computers attempt to figure
>out.  White to play and mate in 9.  (I think.)
>
>kr6/2K5/p7/P7/6PQ/8/8/8 w - - 0 0

[history snipped]
>However, I think that the optimal solution is a mate in 9 in which there
>are slightly different variations, all beginning with 1.g5!  I'll give
>two:
>
>1.g5 Rb7+ 2.Kc6
>
>a) 2...Rb8 3.g6 Ka7 4.Qe7+ Ka8 5.g7 Rc8+ 6.Kb6 Rb8+ 7.Kax6 Rb6+ 8.Kxb6
>Kb8 9.g8=Q#
                     4.Qd4+ will do also
            3.Qh7 will do also

Confirmed.

>b) 2...Kb8 3.Qf4+ Ka8 4.g6 Rb8 5.Qxb8+ Kxb8 6.g7 Ka7 7.Kc7 Ka8 8.g8=Q+
>Ka7 9.Qb8#

Confirmed.

>Try putting this problem on your computer and let me know which solution
>it comes up with and how quickly.

My chess problem solver (it does *not play* chess) completed this one after
211 seconds (3:31) on a not so fast machine.  (No, it's not available.)

And it came up with a second key move for "mate in 9", which is
our old friend 1.Qh1+. It can be improved, again:

>a) 1.Qh1+ Ka7 2.Qh2 Rb7+ 3.Kc8 Rb5 [3...Rb6 4.Qc7+ Ka8 5.axb6 a5 6.Qa7#]
>4.Qc7+ Ka8 5.g5 Rc5 6.g6 Rxc7+ 7.Kxc7 Ka7 8.g7 Ka8 9.g8=Q+ Ka7 10.Qb8#

 4.Qd6 Rb1 5.Qd4+ Ka8 6.Qe4+ Ka7 7.Qxb1 Ka8 8.Qb8#

Better is 3.... Rb3 (all other moves yield shorter mates) with e.g.

 4.Qc7+ Ka8 5.Qc6+ Ka7 6.Qc5+ Ka8 7.Qd5+ Ka7 8.Qxb3 Ka8 9.Qb8#

That was fun!
--
Heiner Marxen   heiner@drb.insel.de     http://www.drb.insel.de/~heiner/

================= end of post 2, start of post 3 =======

Author: Mike Byrne
Email:  chessman.nospam@voicetnet.com.nospam
Date:   1998/12/22
Forums: rec.games.chess.computer

Crafty is programmed to find mate ...it doesn't really care if one is longer
than the other...but on my dual 400 (without tb's and it is important to set
pondering to *off* before eneterneing the position) ..it does find the
mate-in8 ..every move is forced or the mate is even quicker...with tb's
(kppkp) it finds the  *master style*win of
the Qd8 (Mate in 10)


White(1): go
end-game phase
              clearing hash tables
              time surplus   0.00  time limit 1500:00 (1500:00)
              depth   time  score   variation (1)
                9     0.28  10.28   1. Qh1+ Ka7 2. Qc6 Rc8+ 3. Kd7 Rxc6
                                    4. Kxc6 Kb8 5. g5 Kc8 6. g6 Kd8
                9->   1.10  10.28   1. Qh1+ Ka7 2. Qc6 Rc8+ 3. Kd7 Rxc6
                                    4. Kxc6 Kb8 5. g5 Kc8 6. g6 Kd8
               10     1.41  10.32   1. Qh1+ Ka7 2. Qc6 Rc8+ 3. Kd7 Rxc6
                                    4. Kxc6 Kb8 5. g5 Kc8 6. g6 Kd8 7.
                                    g7 Ke7
               10     4.02     ++   1. g5!!
               10->   4.97  10.71   1. g5
               11     5.71     ++   1. g5!!
               11    29.22  Mat10   1. g5 Rb7+ 2. Kd6 Rb8 3. g6 Ka7 4.
                                    Qh7+ Ka8 5. g7 Rc8 6. g8=R Rxg8 7.
                                    Qxg8+ Ka7 8. Qg7+ Ka8 9. Kc6 Kb8 10.
                                    Qb7#
               11    33.16  Mat08   1. Qh1+ Ka7 2. Qh2 Rb7+ 3. Kc8 Rb5
                                    4. Qd6 Rb1 5. Qe7+ Ka8 6. Qe4+ Ka7
                                    7. Qxb1 Ka8 8. Qb8#
               11->  34.21  Mat08   1. Qh1+ Ka7 2. Qh2 Rb7+ 3. Kc8 Rb5
                                    4. Qd6 Rb1 5. Qe7+ Ka8 6. Qe4+ Ka7
                                    7. Qxb1 Ka8 8. Qb8#
              time=34.22  cpu=198%  mat=5  n=16118704  fh=96%  nps=471031
              ext-> checks=3099819 recaps=136 pawns=177186 1rep=507936
              predicted=0  nodes=16118704  evals=80119
              endgame tablebase-> probes done=0  successful=0  maxd=32
              SMP->  split=2080  stop=432  data=9/64  cpu=1:08  elap=34.22



Bernhard Bauer wrote in message <367FA4C2.590128AC@iag.uni-stuttgart.de>...
>
>
>
>
>
>Crafty16.2 on a 2xPPro 233MHz finds the mate in 9 on ply 11.
>The move 1. g5 is found in 6 sec and the whole search is finished in 1:42.
>Here some output from crafty:
>
>               10->   9.08   9.92   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
>                                    Qh7+ Ka8 5. Qc7 Rb1 6. Qd8+ Rb8 7.
>                                    Qxb8+ Kxb8 8. g7 Ka7 9. Kd6
>               11    10.84     ++   1. g5!!
>               11     1:31  Mat09   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
>                                    Qh7+ Ka8 5. g7 Rc8+ 6. Kb6 Rb8+ 7.
>                                    Kxa6 Rb7 8. g8=Q+ Rb8 9. Qb7#
>               11->   1:42  Mat09   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
>                                    Qh7+ Ka8 5. g7 Rc8+ 6. Kb6 Rb8+ 7.
>                                    Kxa6 Rb7 8. g8=Q+ Rb8 9. Qb7#
>              time=1:42  cpu=200%  mat=5  n=30057337  fh=97%  nps=292585
>              ext-> checks=6269540 recaps=290 pawns=277457 1rep=691838
>              predicted=0  nodes=30057337  evals=126326
>              endgame tablebase-> probes done=0  successful=0  maxd=32
>              hashing-> trans/ref=55%  pawn=99%  used=w89% b99%
>              SMP->  split=2340  stop=534  data=9/64  cpu=3:26  elap=1:42
>
>mate in 9 moves.
>
<snip>
>
>Kind regards
>Bernhard
>"Edward D. Collins" wrote:
>
>> Here's a position I still have fun watching computers attempt to figure
>> out.  White to play and mate in 9.  (I think.)
>>
>> kr6/2K5/p7/P7/6PQ/8/8/8 w - - 0 0
>>
>> If memory serves, this was originally published/presented as a mate in
>> 15 with the Queen aesthetically zigzagging up the board...
>>
>> 1.Qh1+ Ka7 2.Qg1+ Ka8 3.Qg2+ Ka7 4.Qf2+ Ka8 5.Qf3+ Ka7 6.Qe3+ Ka7 7.Qe4+
>> Ka8
>> 8.Qd4+ Ka8 9.Qd5+ Ka7 10.Qc5+ Ka8 11.Qc6+ Ka7 12.Qb6+ Rxb6 13.axb6+ Ka8
>> 14.b7+
>> Ka7 15.b8=Q#
>>
>> ... which many programs will still find first, since every move is a
>> check.
>>
>> However, someone, somewhere, actually discovered a two move shorter mate
>> beginning with the non checking move 9.Qd6
>>
>> 1.Qh1+ Ka7 2.Qg1+ Ka8 3.Qg2+ Ka7 4.Qf2+ Ka8 5.Qf3+ Ka7 6.Qe3+ Ka7 7.Qe4+
>> Ka8
>> 8.Qd4+ Ka8 9.Qd6 Rb7+ 10.Kc8 Ka7 11.Qc5+ Ka8 12.Qc6 Ka7 13.Qxb7#
>>
>> Not to be outdone, someone THEN shaved off three more moves from the
>> solution and discovered that at some a mate in 10 was possible...
>> actually a couple of them, each beginning with the subtle 2.Qh2.
>>
>> a) 1.Qh1+ Ka7 2.Qh2 Rb7+ 3.Kc8 Rb5 [3...Rb6 4.Qc7+ Ka8 5.axb6 a5 6.Qa7#]
>> 4.Qc7+ Ka8 5.g5 Rc5 6.g6 Rxc7+ 7.Kxc7 Ka7 8.g7 Ka8 9.g8=Q+ Ka7 10.Qb8#
>>
>> b) 1.Qh1+ Ka7 2.Qh2 Rb7+ 3.Kc8 Rb5 4.Qc7+ Ka8 5.Qc6+ Ka7 6.Kc7 Rb1
>> 7.Qb6+
>> Rxb6 8.axb6+ Ka8 9.b7+ Ka7 10.b8=Q#
>>
>> However, I think that the optimal solution is a mate in 9 in which there
>> are slightly different variations, all beginning with 1.g5!  I'll give
>> two:
>>
>> 1.g5 Rb7+ 2.Kc6
>>
>> a) 2...Rb8 3.g6 Ka7 4.Qe7+ Ka8 5.g7 Rc8+ 6.Kb6 Rb8+ 7.Kax6 Rb6+ 8.Kxb6
>> Kb8 9.g8=Q#
>>
>> b) 2...Kb8 3.Qf4+ Ka8 4.g6 Rb8 5.Qxb8+ Kxb8 6.g7 Ka7 7.Kc7 Ka8 8.g8=Q+
>> Ka7 9.Qb8#
>>
>> Try putting this problem on your computer and let me know which solution
>> it comes up with and how quickly.
>>
>> - Ed
>
================ end of post 3, start of post 4 ===========

Author:

               Heiner Marxen
Email:
               heiner@DrB.Insel.DE
Date:
               1998/12/22
Forums:
               rec.games.chess.computer
more headers



               author profile

                 view thread



In article <1JQf2.500$7P2.6731730@news2.voicenet.com>,
Mike Byrne <chessman.nospam@voicetnet.com.nospam> wrote:
[snip]
>               11->  34.21  Mat08   1. Qh1+ Ka7 2. Qh2 Rb7+ 3. Kc8 Rb5
>                                    4. Qd6 Rb1 5. Qe7+ Ka8 6. Qe4+ Ka7
>                                    7. Qxb1 Ka8 8. Qb8#

According to my program, 3... Rb3 cannot be completed as Mat08.
If crafty is right, my program is wrong <sniff>.  So, could you
please ask your crafty to show a solution to that move?

See also my direct followup.
--
Heiner Marxen   heiner@drb.insel.de     http://www.drb.insel.de/~heiner/
============ end of post 4, start of post 5 =======

Author: Mike Byrne
Email:  @voicetnet.com.nospam
Date:   1998/12/26
Forums:  rec.games.chess.computer

I just added a new mate finder command  for crafty <which I have passed on
Bob Hyatt>

this is a mate in 9:

 18->   6:41  Mat09   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
                      Qe7+ Ka8 5. g7 Rc8+ 6. Kb6 Rb8+ 7.
                      Kxa6 Rb6+ 8. axb6 Kb8 9. g8=Q#
time=6:41  cpu=199%  mat=5  n=205570273  fh=98%  nps=512324
ext-> checks=30969436 recaps=1748 pawns=3984818 1rep=8931601
predicted=0  nodes=205570273  evals=247991
endgame tablebase-> probes done=0  successful=0  maxd=32
SMP->  split=7122  stop=451  data=10/64  cpu=13:20  elap=6:41
Mike Byrne wrote in message ...
>>               11->   1:42  Mat09   1. g5 Rb7+ 2. Kc6 Rb8 3. g6 Ka7 4.
>>                                    Qh7+ Ka8 5. g7 Rc8+ 6. Kb6 Rb8+ 7.
>>                                    Kxa6 Rb7 8. g8=Q+ Rb8 9. Qb7#
>>              time=1:42  cpu=200%  mat=5  n=30057337  fh=97%  nps=292585
>>              ext-> checks=6269540 recaps=290 pawns=277457 1rep=691838
>>              predicted=0  nodes=30057337  evals=126326
>>              endgame tablebase-> probes done=0  successful=0  maxd=32
>>              hashing-> trans/ref=55%  pawn=99%  used=w89% b99%
>>              SMP->  split=2340  stop=534  data=9/64  cpu=3:26  elap=1:42
>>
>>mate in 9 moves.
>>
>>
>>Kind regards
>>Bernhard
>>"Edward D. Collins" wrote:
>>
>>> Here's a position I still have fun watching computers attempt to figure
>>> out.  White to play and mate in 9.  (I think.)
>>>
>>> kr6/2K5/p7/P7/6PQ/8/8/8 w - - 0 0
>>>

>>>
>>> However, I think that the optimal solution is a mate in 9 in which there
>>> are slightly different variations, all beginning with 1.g5!  I'll give
>>> two:
>>>
>>> 1.g5 Rb7+ 2.Kc6
>>>
>>> a) 2...Rb8 3.g6 Ka7 4.Qe7+ Ka8 5.g7 Rc8+ 6.Kb6 Rb8+ 7.Kax6 Rb6+ 8.Kxb6
>>> Kb8 9.g8=Q#
>>>
>>> b) 2...Kb8 3.Qf4+ Ka8 4.g6 Rb8 5.Qxb8+ Kxb8 6.g7 Ka7 7.Kc7 Ka8 8.g8=Q+
>>> Ka7 9.Qb8#
>>>
>>> Try putting this problem on your computer and let me know which solution
>>> it comes up with and how quickly.
>>>
>>> - Ed
>>
>
>



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.