Author: Mike Hood
Date: 18:00:53 02/08/03
Today I set up a position to compare the EGTB probing efficiency of several Chessbase engines. My PC is a Pentium III 733 MHz and I have all the 3/4/5-piece EGTBs. For the tests I gave the engines 128 MB hash tables. The position is: [D] 8/6k1/6P1/p4K2/8/8/4P1P1/8 w - - 0 1 The optimal win is Mate in 25 with minor variations in the moves, for instance 1. Ke5 a4 2. Kd4 a3 3. Kc3 a2 4. Kb2 a1Q+ 5. Kxa1 (Here it enters the KPP-KP EGTB) 5...Kxg6 6. Kb2 Kf5 7. Kb3 Kf4 8. Kc4 Ke4 9. g4 Ke5 10. Kc5 Ke6 11. Kd4 Kf6 12. e4 Kg5 13. e5 Kg6 14. e6 Kf6 15. Kd5 Ke7 16. g5 Ke8 17. g6 Kf8 18. Ke5 Ke8 19. g7 Ke7 20. Kd5 Kf6 21. g8Q Kf5 22. e7 Kf4 23. e8Q Kf3 24. Qe4+ Kf2 25. Qgg2# 1-0 The purpose of the test was not only to find the best first move (Ke5 or Ke4), but to recognize that this move is Mate in 25. This should be an easy task for any engine with EGTB probing, but almost impossible for an engine without probing. For instance, Fritz 5.32 finds the correct line, but after 30 minutes only gives a score of +5.84 I ran the test with all the engines that use EGTB probing, and I stopped the test either (1) when the correct result was found or (2) after 5 minutes. Of the 21 engines I tested only 11 found the correct result. Correct Results: Engine Score Time Depth Nodes tb= Line Shredder 6 #25 0 11/22 17kN 1864 1.Ke4 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1B+ 5.Kxa1 Kxg6 BamBam #25 1 14 760kN 58165 1.Ke4 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 InmiChess 3.06 #25 1 13 19kN 1145 1.Ke4 a4 2.Kd3 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Fritz 7 #25 1 13/26 714kN 484 1.Ke5 a4 2.Kd4 a3 Crafty 19.01 #25 2 10/12 18kN 335 1.Ke5 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Hiarcs 7.32 #25 2 11/25 154kN 487 1.Ke5 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Shredder 7 #25 2 12/12 27kN 2311 1.Ke5 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1B+ 5.Kxa1 Kxg6 Fritz 8 #25 3 13/27 1258kN 2744 1.Ke4 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Kxg6 6.Kb2 Kf5 Goliath Light 1.5 #25 3 12/21 722kN 3078 1.Ke4 a4 2.Kd3 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Gromit 3.1 #25 3 11/52 61kN 1404 1.Ke4 Kxg6 Hiarcs 8 #25 5 11/25 601kN 2232 1.Ke4 a4 2.Kd3 All the engines that find the correct answer need less than 5 seconds, even on my slow computer. Although I can't vouch for the scientific accuracy, it seems that Shredder 6, Fritz 7 and Hiarcs 7.32 are all faster than their newer versions. Incorrect Results: Engine Score Time Depth Nodes tb= Line Junior 7 #23 12 21 1788kN 13585 1.Ke4 Kxg6 SOS #23 54 17/33 11995kN 29437 1.Ke4 Kxg6 Chess Tiger 14 #24 3 13 767kN 1101 1.Ke4 Kxg6 AnMon 5.07 #26 5 9 136kN 1440 1.Ke4 a4 Exchess 4.02 #24 4 11 786kN unknown 1.Ke4 a4 2.Kd3 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Patzer 3.11 #24 1 10 54kN 1544 1.Ke5 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 a1R 5.Kxa1 The Crazy Bishop #29 3:19 17 86975kN unknown 1.Ke4 a4 2.Kd3 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Kf6 6.g7 Ke5 7.g8Q Kd6 8.Qe6+ Ikarus 0.18 #30 1:40 16/30 18302kN 105079 1.Ke5 a4 2.Kd4 a3 3.Kc3 a2 4.Kb2 Kf6 5.g7 a1R 6.Kxa1 Ke5 7.g8Q Comet B55 #75 1:26 16/31 26455kN 25065 1.Ke4 Kf8 2.g7+ Kf7 3.Kd4 Nimzo 8 3.96 1:20 20/40 52331kN 2210 1.Ke4 a4 2.Kd3 a3 3.Kc3 a2 4.Kb2 a1Q+ 5.Kxa1 Kg8 6.Kb2 Junior, Chess Tiger and SOS all make the same error of expecting 1...Kxg6 (Mate in 23) instead of the best reply 1...a4 (Mate in 25). Chess Tiger makes the additional error of scoring this line as Mate in 24. AnMon, Exchess and Patzer find the correct line, but the score is wrong. The Crazy Bishop finds the correct line as far as the tablebase entry point 5.Kxa1, but chooses an inferior path afterwards. Ikarus and Comet fail to find the correct line. Nimzo 8 is the biggest failure. Even though it probes the endgame tablebases, it doesn't seem to use the probe results correctly, so it works even worse than the engines with no probing. I deliberately confined myself to testing Chessbase engines. I wanted to leave the other forum members to test their favourite Winboard and UCI engines themselves :)
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.