Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Testing evaluation quality with test suits

Author: Albert Bertilsson

Date: 23:10:19 12/14/03

Go up one level in this thread


On December 14, 2003 at 18:20:34, scott farrell wrote:

>On December 14, 2003 at 10:25:12, Albert Bertilsson wrote:
>
>>Hi!
>>
>>I've started to look at the quality of the evaluation in Sharper and have some
>>questions regarding a test suite I've found.
>>
>>The test suite is called QuietTest and has 81660 positions, from the name and
>>the number of positions I guessed that this test should test the q-search
>>function. So I tested it with q-search only and Sharper "solves" about 13% of
>>the positions.
>>
>
>Sounds interesting, where can I download the whole lot from?
>
>Scott

I assume I found it on Dann's ftp, he's got everything! If you can't find it let
me know and I'll pack it and mail it to you (If your account can handle multiple
MB mails).

/Regards Albert

>
>>My questions are:
>>1. Is the test supposed to be run like I run it?
>>2. What does your engine score in the test?
>>
>>If you don't have the test here is a sample of 100 positions, Sharper solves 11
>>of them, what is your score?
>>
>>1B1b1k2/5ppp/pp2p3/3p4/3P4/P1N1P3/1Pb2PPP/6K1 w - - bm Kf1; id
>>"QuietTest.00001";
>>1B1b2k1/5ppp/pp2p3/3p4/3P4/P1N1P3/1Pb2PPP/6K1 b - - bm Kf8; id
>>"QuietTest.00002";
>>1B1b2k1/5ppp/pp2p3/3p4/3P4/P1NbP3/1P3PPP/6K1 w - - bm Ba7; id "QuietTest.00003";
>>1B1b2k1/p4ppp/1p2p3/3p4/3P4/P1NbP3/1P3PPP/6K1 b - - bm a6; id "QuietTest.00004";
>>1B1b4/3k1ppp/pp2p3/3p4/2bP1N2/P3P3/1P1K1PPP/8 b - - bm g5; id "QuietTest.00005";
>>1B1b4/3k1ppp/pp2p3/3p4/3P4/P1N1P3/1Pb1KPPP/8 w - - bm Kd2; id "QuietTest.00006";
>>1B1b4/3k1ppp/pp2p3/3p4/3P4/Pb2P3/1P1KNPPP/8 b - - bm Bc4; id "QuietTest.00007";
>>1B1b4/3k1ppp/pp2p3/3p4/3P4/PbN1P3/1P1K1PPP/8 w - - bm Ne2; id "QuietTest.00008";
>>1B1b4/3k3p/bp2pp2/p2p2p1/3P4/PPKNP3/5PPP/8 w - - bm Bg3; id "QuietTest.00009";
>>1B1b4/3k3p/pp2pp2/3p2p1/2bP4/P1KNP3/1P3PPP/8 b - - bm a5; id "QuietTest.00010";
>>1B1b4/4kppp/pp2p3/3p4/3P4/P1N1P3/1Pb1KPPP/8 b - - bm Kd7; id "QuietTest.00011";
>>1B1b4/4kppp/pp2p3/3p4/3P4/P1N1P3/1Pb2PPP/5K2 w - - bm Ke2; id "QuietTest.00012";
>>1B1bn3/3b2kp/3p2p1/1p1Pp3/4Pp2/2PB1P1P/6P1/2N2K2 w - - bm Ba7; id
>>"QuietTest.00013";
>>1b1q1rk1/5p1p/p5p1/3B4/4PP2/2Q5/P4P1P/5RK1 w - - bm Qf3; id "QuietTest.00014";
>>1b1q1rk1/5p1p/p5p1/3B4/4PP2/5Q2/P4P1P/5RK1 b - - bm Qh4; id "QuietTest.00015";
>>1B1Q4/5ppk/p3pn1p/P7/4P1P1/5P1P/1q4K1/8 w - - bm Kg1; id "QuietTest.00016";
>>1b1r1n2/5k2/R3p2p/2B3pP/4B1P1/5P2/6K1/8 w - - bm Ra8; id "QuietTest.00017";
>>1b1r1rk1/1p1qnp1p/p1p3p1/5p2/1P1P2b1/P3P1P1/2QN1P1P/R1R1NBK1 b - - bm g5; id
>>"QuietTest.00018";
>>1b1r1rk1/1p1qnp1p/p1p3p1/5p2/1P1P2b1/P3P3/2QN1PPP/R1R1NBK1 w - - bm g3; id
>>"QuietTest.00019";
>>1b1r1rk1/1p2qppp/p1n1p1b1/8/3Pp1B1/1QB1P1PP/PP3PN1/2RR2K1 b - - bm Rd5; id
>>"QuietTest.00020";
>>1b1r1rk1/pp2qppp/2p1pn2/8/2PP4/1Q2BBPP/PP3P2/R2R2K1 b - - bm Rd7; id
>>"QuietTest.00021";
>>1b1r1rk1/pp2qppp/2p1pn2/8/2PP4/1Q3BPP/PP3P2/R1BR2K1 w - - bm Be3; id
>>"QuietTest.00022";
>>1b1r2k1/1p1r1ppp/p5q1/3p1b2/3Q1P2/1P2P1P1/PB4BP/2R2RK1 w - - bm g4; id
>>"QuietTest.00023";
>>1b1r2k1/1p1r1ppp/p5q1/3p1b2/3Q1PP1/1P2P3/PB4BP/2R2RK1 b - - bm Bd3; id
>>"QuietTest.00024";
>>1b1r2k1/1p1r1ppp/p5q1/3p4/3Q1PP1/1P1bP3/PB4BP/2R2RK1 w - - bm Rf2; id
>>"QuietTest.00025";
>>1b1r2k1/1p2qpp1/2r1b2p/1N1p4/P2Bn3/4P2P/B3QPP1/2RR2K1 w - - bm Ba7; id
>>"QuietTest.00026";
>>1b1r2k1/1q3pp1/2p1n3/2n1p1p1/1NQ1P1P1/1P2B2P/5PB1/5R1K b - - bm Ba7; id
>>"QuietTest.00027";
>>1b1r2k1/pp1rqpp1/2p1pn1p/8/2PP4/1Q2BBPP/PP1R1P2/3R2K1 w - - bm a3; id
>>"QuietTest.00028";
>>1b1r2k1/pp1rqppp/2p1pn2/8/2PP4/1Q2BBPP/PP1R1P2/3R2K1 b - - bm h6; id
>>"QuietTest.00029";
>>1b1rr1k1/1q3pp1/2p1n3/2n1p1p1/1NQ1P1P1/1PB4P/3R1PB1/5R1K b - - bm Rxd2; id
>>"QuietTest.00030";
>>1b1rr1k1/1q3pp1/2p1n3/2n1p1p1/1NQ1P1P1/1PB4P/5PB1/3R1R1K w - - bm Rd2; id
>>"QuietTest.00031";
>>1B2kbr1/4rp1p/pq6/8/1p6/5Q2/PPP3PP/R4RK1 w - - bm Kh1; id "QuietTest.00032";
>>1b2n3/1P1qkpp1/8/Q3p2p/P6P/4P1P1/1R3P2/6K1 b - - bm Qc6; id "QuietTest.00033";
>>1b2n3/1P2kpp1/2q5/Q3p2p/P6P/4P1P1/1R3P2/6K1 w - - bm Qb4+; id "QuietTest.00034";
>>1B2n3/3b2kp/3p2p1/1p1Pp3/4Pp1b/2PB1P1P/4N1P1/5K2 w - - bm Nc1; id
>>"QuietTest.00035";
>>1b2r1k1/1b1n2np/pq2p1p1/1p1p4/1B1P4/1Q3N2/PP2NPPP/1B2R1K1 w - - bm Bd2; id
>>"QuietTest.00036";
>>1b2r1k1/1b1n2np/pq2p1p1/1p1p4/3P4/1Q3N2/PP1BNPPP/1B2R1K1 b - - bm Rf8; id
>>"QuietTest.00037";
>>1b2r1k1/1bq3np/p3pnp1/1pBp2N1/3P1P2/1Q6/PP2N1PP/1B2R1K1 b - - bm Nf5; id
>>"QuietTest.00038";
>>1b2r1k1/1bq3np/p3pnp1/1pBp2N1/3P4/1Q6/PP2NPPP/1B2R1K1 w - - bm f4; id
>>"QuietTest.00039";
>>1b2r1k1/1bq4p/p3pnp1/1pBp1nN1/3P1P2/1Q6/PP2N1PP/1B2R1K1 w - - bm Qh3; id
>>"QuietTest.00040";
>>1b2r1k1/1bq4p/p3pnp1/1pBp1nN1/3P1P2/7Q/PP2N1PP/1B2R1K1 b - - bm Bc8; id
>>"QuietTest.00041";
>>1b2r1k1/1q3pp1/2p1n3/2n1p1p1/1NQ1P1P1/1P5P/3B1PB1/5R1K b - - bm Rd8; id
>>"QuietTest.00042";
>>1b2r1k1/4r2p/2p2p2/1p1p1Bp1/p2P4/P1P5/1P1BbPPP/R3R2K b - - bm Kg7; id
>>"QuietTest.00043";
>>1b2r1k1/4r2p/2p2p2/1p1p2p1/p2P4/P1PB4/1P1BbPPP/R3R2K w - - bm Bf5; id
>>"QuietTest.00044";
>>1b2r1k1/r6p/2p2p2/1p1p2p1/p2P2b1/P1PB4/1P1B1PPP/R4R1K b - - bm Be2; id
>>"QuietTest.00045";
>>1b2r1k1/r6p/2p2p2/1p1p2p1/p2P2b1/P1PB4/1P3PPP/R1B2R1K w - - bm Bd2; id
>>"QuietTest.00046";
>>1b2r1k1/r6p/2p2p2/1p1p2p1/p2P4/P1PB4/1P1BbPPP/R4R1K w - - bm Rfe1; id
>>"QuietTest.00047";
>>1b2r3/1p1b1k2/1Pp3pp/2Pp1p2/3NnP1q/rN1BPR1P/2Q3P1/6RK w - - bm Bxe4; id
>>"QuietTest.00048";
>>1b2r3/1p1bqk2/1Pp3pp/2Pp1p2/3NnP2/rN1BPR1P/2Q3P1/2R4K w - - bm Rg1; id
>>"QuietTest.00049";
>>1b2r3/1p1bqk2/1Pp3pp/2Pp1p2/3NnP2/rN1BPR1P/2Q3P1/6RK b - - bm Qh4; id
>>"QuietTest.00050";
>>1b2r3/1p1bqk2/1Pp3pp/2Pp1p2/r2NnP2/1N1BPR1P/1Q4P1/2R4K w - - bm Qc2; id
>>"QuietTest.00051";
>>1b2r3/1p1bqk2/1Pp3pp/2Pp1p2/r2NnP2/1N1BPR1P/2Q3P1/2R4K b - - bm Ra3; id
>>"QuietTest.00052";
>>1b2r3/4r1k1/2p2p2/1p1p1Bpp/p2P4/P1P5/1P1BbPPP/R3R1K1 w - - bm g3; id
>>"QuietTest.00053";
>>1b2r3/4r1kp/2p2p2/1p1p1Bp1/p2P4/P1P5/1P1BbPPP/R3R1K1 b - - bm h5; id
>>"QuietTest.00054";
>>1b2r3/4r1kp/2p2p2/1p1p1Bp1/p2P4/P1P5/1P1BbPPP/R3R2K w - - bm Kg1; id
>>"QuietTest.00055";
>>1B3bk1/4p2p/p4pp1/P2b4/3P4/4PN1P/2r2PP1/R5K1 b - - bm e6; id "QuietTest.00056";
>>1B3bk1/7p/p3ppp1/P2b4/3P4/4PN1P/2r2PP1/1R4K1 b - - bm Rc4; id "QuietTest.00057";
>>1b3k1r/p3pp1p/6p1/1B1P4/4P3/5P2/P4P1P/4K2R w K - bm O-O; id "QuietTest.00058";
>>1b3k1r/p3pp1p/6p1/1B1P4/4P3/5P2/P4P1P/5RK1 b - - bm Bf4; id "QuietTest.00059";
>>1b3n2/3r1k2/R3p2p/2B3pP/4B1P1/5P2/6K1/8 b - - bm Rd8; id "QuietTest.00060";
>>1b3r1k/5ppp/pq6/1p1R4/1P6/P4N1P/3Q1PP1/6K1 w - - bm Ne5; id "QuietTest.00061";
>>1b3rk1/1b1n2np/pq2p1p1/1p1p2N1/3P4/1Q6/PP1BNPPP/1B2R1K1 b - - bm Nf6; id
>>"QuietTest.00062";
>>1b3rk1/1b4np/pq2pnp1/1p1p2N1/1B1P4/1Q6/PP2NPPP/1B2R1K1 b - - bm Re8; id
>>"QuietTest.00063";
>>1b3rk1/1b4np/pq2pnp1/1p1p2N1/3P4/1Q6/PP1BNPPP/1B2R1K1 w - - bm Bb4; id
>>"QuietTest.00064";
>>1b3rk1/1p1q2pp/p1n1p3/3r1p2/3Pp3/4P1PP/PP1RQPN1/2R1B1K1 b - - bm Nb4; id
>>"QuietTest.00065";
>>1b3rk1/1p1q2pp/p1n1p3/3r1p2/3Pp3/4P1PP/PP2QPN1/2RRB1K1 w - - bm Rd2; id
>>"QuietTest.00066";
>>1b3rk1/1p1q2pp/p3p3/3r1p2/1n1Pp3/4P1PP/PP1RQPN1/2R1B1K1 w - - bm a3; id
>>"QuietTest.00067";
>>1b3rk1/1p2q1pp/p1n1p3/3r1p2/3Pp3/2B1P1PP/PP2QPN1/2RR2K1 w - - bm Be1; id
>>"QuietTest.00068";
>>1b3rk1/1p2q1pp/p1n1p3/3r1p2/3Pp3/4P1PP/PP2QPN1/2RRB1K1 b - - bm Qd7; id
>>"QuietTest.00069";
>>1b3rk1/1p2qppp/p1n1p1b1/3r4/3Pp1B1/1QB1P1PP/PP3PN1/2RR2K1 w - - bm Be2; id
>>"QuietTest.00070";
>>1b3rk1/1p2qppp/p1n1p1b1/3r4/3Pp3/1QB1P1PP/PP2BPN1/2RR2K1 b - - bm Bh5; id
>>"QuietTest.00071";
>>1b3rk1/1p2qppp/p1n1p3/3r3b/3Pp3/1QB1P1PP/PP2BPN1/2RR2K1 w - - bm Qc2; id
>>"QuietTest.00072";
>>1b3rk1/1p2qppp/p1n1p3/3r4/3Pp3/2B1P1PP/PP2QPN1/2RR2K1 b - - bm f5; id
>>"QuietTest.00073";
>>1b3rk1/pp1rqppp/2p1pn2/8/2PP4/1Q2BBPP/PP1R1P2/R5K1 b - - bm Rfd8; id
>>"QuietTest.00074";
>>1b3rk1/pp1rqppp/2p1pn2/8/2PP4/1Q2BBPP/PP3P2/R2R2K1 w - - bm Rd2; id
>>"QuietTest.00075";
>>1b4k1/1b1q3p/p4pp1/3pn3/1B1N3P/1P2PPP1/3Q1K2/5B2 b - - bm Nc6; id
>>"QuietTest.00076";
>>1b4k1/1b5p/p4pp1/3p4/7P/1PB1PPP1/5K2/5B2 b - - bm f5; id "QuietTest.00077";
>>1b4k1/1b5p/p5p1/3p1p2/7P/1PBBPPP1/5K2/8 b - - bm Bc8; id "QuietTest.00078";
>>1B4k1/1p3pp1/6p1/1P1n4/8/b6P/4BPP1/6K1 b - - bm Bc5; id "QuietTest.00079";
>>1B4k1/1p3pp1/6p1/1Pbn4/8/7P/4BPP1/6K1 w - - bm Be5; id "QuietTest.00080";
>>1B4k1/1p3pp1/6p1/3n4/1P6/b6P/4BPP1/6K1 w - - bm b5; id "QuietTest.00081";
>>1B4k1/4p1bp/p4pp1/3b4/P2P4/4PN1P/2r2PP1/R5K1 b - - bm Bf8; id "QuietTest.00082";
>>1b4k1/5pp1/Pp6/1Bn5/2PB3p/1p3P2/4r1P1/R6K w - - bm Rb1; id "QuietTest.00083";
>>1B4k1/5pp1/pQ2p2p/P6n/2q1P3/5PPP/8/6K1 b - - bm Qc3; id "QuietTest.00084";
>>1B4k1/p3p1bp/5pp1/3b4/P2P4/4PN2/2r2PPP/R5K1 b - - bm a6; id "QuietTest.00085";
>>1b6/1p1b1k2/1Pp3pp/2Pp1p2/3NrP1q/rN2PR1P/1Q4P1/6RK b - - bm Ra4; id
>>"QuietTest.00086";
>>1b6/1p1b1k2/1Pp3pp/2Pp1p2/r2NrP1q/1N2PR1P/1Q4P1/R6K b - - bm Rb4; id
>>"QuietTest.00087";
>>1b6/1P2kpp1/2qn4/P3p2p/1Q5P/4P1P1/1R3P2/6K1 b - - bm Ke6; id "QuietTest.00088";
>>1b6/1P3pp1/2qnk3/P3p2p/1Q5P/4P1P1/1R3P2/6K1 w - - bm Qb3+; id "QuietTest.00089";
>>1B6/3k2p1/p1p2b1p/4p2R/P1R5/2P3P1/r4PKP/r7 w - - bm Re4; id "QuietTest.00090";
>>1b6/4n3/1pn1p1kp/p2p1pp1/P2P1PP1/1P1NPBBP/5K2/8 w - - bm Kg2; id
>>"QuietTest.00091";
>>1b6/4n3/1pn1pk1p/p2p1pp1/P2P1PP1/1P1NPBBP/4K3/8 w - - bm Kf2; id
>>"QuietTest.00092";
>>1b6/4n3/1pn1pk1p/p2p1pp1/P2P1PP1/1P1NPBBP/5K2/8 b - - bm Kg6; id
>>"QuietTest.00093";
>>1b6/8/5k2/pb1p2p1/3P2P1/1P1N1N2/3K4/8 w - - bm Ke3; id "QuietTest.00094";
>>1Bb1kb1r/5ppp/p1p1pn2/2q5/8/2N3P1/PPP2PBP/R2Q2K1 b k - bm Be7; id
>>"QuietTest.00095";
>>1bb1r1k1/1p3ppp/r1q5/1N1p4/P1nBn1BN/4P1PP/5PK1/R3Q2R b - - bm f5; id
>>"QuietTest.00096";
>>1bb1r1k1/1p3ppp/r1q5/1N1p4/P1nBn2N/4P1PP/4BPK1/R3Q2R w - - bm Bg4; id
>>"QuietTest.00097";
>>1bb1r1k1/2q4p/p3pnp1/1pBp1nN1/3P1P2/7Q/PP2N1PP/1B2R1K1 w - - bm g4; id
>>"QuietTest.00098";
>>1bb1r1k1/6qp/p2npnp1/1pBp2N1/3P1PP1/7Q/PP2N2P/1BR3K1 w - - bm Nf3; id
>>"QuietTest.00099";
>>1bb1r1k1/6qp/p2npnp1/1pBp4/3P1PP1/5N1Q/PP2N2P/1BR3K1 b - - bm Nde4; id
>>"QuietTest.00100";
>>
>>/Regards Albert



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.