Computer Chess Club Archives


Search

Terms

Messages

Subject: Revised ECM Test Suite (some additional analysis needed)

Author: Omid David Tabibi

Date: 04:31:35 07/19/04


It is well known that the ECM test suite has a large number of erros and
inaccuracies. GCP has compiled a test suite which doesn't include the more
problematic positions. But I wanted to take a look at all 879 positions of ECM
and correct them.

On my AMD64 3200+, I ran Junior 8, Deep Fritz 8, Shredder 8, and Falcon for 10
seconds per position. Results were:

Shredder 8:    696
Deep Fritz 8:  696
Falcon:        680
Junior 8:      637

Next, I analyzing all the positions were all the engines failed to solve (about
100 positions). The resived test suite can be downloaded from:

http://www.cs.biu.ac.il/~davoudo/pubs/ecmx.epd

Each of these 100 positions was of one of the following types:

1. Correct move, but requires very deep search. Naturally I didn't change
anything.

2. Correct move, but there are additional winning moves. I added the other
winning moves. Sometimes the alternative winning moves were much better, but I
didn't remove the original move, as long as it was not a blunder.

3. Blunder or bad moves. Some suggested best moves are simply blunders. It seems
that these moves have been introduced as a result of typos (e.g. Bd4 instead of
Bc4). In some other cases the best move is wrong because the defending side has
a nice (and usually deep) defense, apparently not taken into consideration when
compiling the test suite.

4. I didn't find any win for the suggested best move, or am not sure whether it
is indeed the best move. There are 26 such positions which appear below. It
would be nice if you could analyze them longer and see whether the suggested
best move is indeed the best, or alternatives should be added:

r3r1k1/pp3ppp/1qp5/2bbNQ2/5B2/2P5/PP3PPP/3RR1K1 w - - bm Rxd5; id "ECM.349";
1k2rb1r/pbp1q1pp/2pp4/2PnpP2/Q7/1P3NP1/PB5P/2KR1B1R w - - bm Rxd5; id "ECM.371";
2bq1b2/1p3k2/1r1p1p1p/pNpP1PpP/P1P3P1/6B1/1P5Q/2K1R3 w - - bm Re6; id "ECM.396";
rq1r2k1/3nbp1p/p2p2p1/1p1Pp1P1/3B1P2/3B3Q/PPP4P/2KR2R1 w - - bm f5; id
"ECM.497";
r3k1r1/pb1qpp1p/1pnp1n1B/2p2R2/3PP3/2P3PN/P1P3BP/R2Q3K b q - bm Nxe4; id
"ECM.561";
8/p5k1/2p2p2/4p1p1/1p2P3/1P4PP/1PP5/5K2 b - - bm g4; id "ECM.601";
8/8/4b1p1/2Bp3p/5P1P/1pK1Pk2/8/8 b - - bm g5; id "ECM.605";
4r2k/3n1Qpp/1pRP1p2/p4P2/1p1P2P1/6rP/P3q1B1/6RK w - - bm Qe6; id "ECM.624";
r1b2rk1/2q1bppp/p1n1p3/1p1np1P1/5P2/1NNBBQ2/PPP4P/R4RK1 w - - bm Bxh7+; id
"ECM.665";
2rqr1k1/1p1bbppp/p3p3/n7/3P2Q1/2PB1N2/P4PPP/R1B1R1K1 w - - bm Bxh7+; id
"ECM.673";
r1b2r1k/ppppq1pp/2n1n3/6N1/2B2P2/4B3/PPP3PP/R2Q1RK1 w - - bm Nxh7; id "ECM.674";
rn1q1rk1/pb1p1ppp/4p3/2pnP3/1bp5/2N2N2/PP3PPP/RBBQK2R w KQ - bm Bxh7+; id
"ECM.675";
r1b2rk1/1pq2ppp/p1p1p3/2n1P3/2N2P2/3B4/PPP3PP/R2Q1RK1 w - - bm Bxh7+; id
"ECM.677";
2k3r1/ppq2p1p/n1Pb1p2/8/6r1/4BN1b/PPQ2PPP/R3RBK1 b - - bm Bxh2+; id "ECM.698";
r5k1/pn1q1rpp/2pp4/5R1N/bP6/4BQ2/P4PPP/2R3K1 w - - bm Nxg7; id "ECM.714";
4kbr1/pp1bpp1p/3p1p2/3N4/1PBQP3/q7/P1r2PPP/R4RK1 b - - bm Rxg2+; id "ECM.725";
r3r1k1/pbq1ppbp/1pp2np1/4N3/3P4/2P5/PPB1QPPP/R1B1R1K1 w - - bm Nxf7; id
"ECM.762";
2rqr1k1/pb2bp1p/1pn1pnpB/4N3/3P4/P1N3R1/1P3PPP/RB1Q2K1 w - - bm Rxg6+; id
"ECM.810";
1q1r3k/3P1pp1/ppBR1n1p/4Q2P/P4P2/8/5PK1/8 w - - bm Rxf6; id "ECM.825";
rr4k1/5p1p/n2p1npQ/q1pP4/6P1/2N2P1N/PP1R3P/2K4R b - - bm Rxb2; id "ECM.853";
5rk1/5rp1/p2qb3/R2Np2Q/3bP2B/7P/P5PK/1R6 b - - bm Rf1; id "ECM.856";
3r1rk1/p3q1pp/2Rb4/3P4/8/1P2Qp2/PB3PPP/3R2K1 b - - bm Bxh2+; id "ECM.858";
8/2q1np1k/6p1/1PPn3p/8/P2Q3P/6P1/2R4K b - - bm Nf5; id "ECM.864";
7R/1p1r4/2b2p2/2Pp1qk1/3P1bp1/3N4/2Q2PB1/6K1 w - - bm Rg8+; id "ECM.866";
r6r/1pn1b1k1/2p1pq2/2Pp3p/p2P1Pp1/P2BP1P1/1P2QB2/3R2KR w - - bm Rxh5; id
"ECM.867";
6k1/1R4pp/2p5/8/P1rNp3/6Pb/4PK2/8 w - - bm a5; id "ECM.878";







This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.