Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Anyone have any experience with the COVAX testsuites?

Author: Dann Corbit

Date: 21:18:55 12/19/05

Go up one level in this thread


On December 19, 2005 at 13:21:26, paul bedrey wrote:

>I'm picking up where I left off about 1.5 years ago in cobbing together a large
>POSITIONAL testsuite. I've got both COVAX 1 and 3 marked for inclusion but its
>been so long I can't remember any of the specifics. I think it was supposed to
>be tested at 3 or 5 minutes. Anyway Rybka beta doe not agree with some of the
>answers so I'm wondering if they are too subjective. Any opinions?

I am doing a COVAX run right now.

From:
http://sccs.muldermedia.de/index.php?section=suites&subsection=covax

Covax test suite results
Covax 1 suite results
Covax 2 suite results
Complete Covax suite results
Instructions on how to run the suites Covax 1 suite (10 min/position):
Positions Program  CPU  Hash  Solved pos.
7  Shredder II  Pentium II 300    #1, 3, 10, 14, 21, 22, 25
6  Junior 4.3  Pentium 166
5  CSTal v.268  K6-200
4  Ferret v.281  Pentium Pro 200
 Crafty 12.08  K6-233
 Diep 1.57.20  Pentium 133
 Phalanx VI  Pentium 133
2  MChess Pro 7.0  Pentium Pro 200*  60 MB  #10, 22
 Rebel 9.0  Pentium Pro 200*  60 MB  #3, 22
 Junior 4.6  Pentium 200 MMX  32 MB
 MChess Pro 6.0  Pentium 133
 Fritz 5.0  Pentium 166
 Rebel 8.0  Pentium 120
 Virtual Chess 1.02  Pentium 150
 Gromit 1.2  486-100
 Goliath Chess 1.1  K5-133
 Phalanx V  Pentium Pro 200
1  Hiarcs 6.0 (engine)  Pentium Pro 200*  32 MB  #9
 Hiarcs 6.0  Pentium 133
 Crafty 12.7  Pentium 120
 MacChess 3.0  7300-180
0  Crafty 11.21  SGI 02 R10k/150
 KnowChess 7.23.97  Cyrix P-166
* results with a star in the CPU column are by me

Covax 2 suite (5 min/position):
Positions Program  CPU  Hash  Solved  Comment
2  Hiarcs 6.0 (engine)  Pentium Pro 200*  32 MB  #1, 10
 Nimzo 7.32  Pentium Pro 233*  32 MB  #1,13
 Hiarcs 7.0  Pentium Pro 200*  96 KB  #10, 13
 Rebel 8.0  Pentium 150    #1, 10
1  Rebel 9.0  Pentium Pro 200*  60 MB  #10  took 5:08 for #1
 Junior 5.0  Pentium Pro 200*  32 MB  #10
 MChess Pro 7.0  Pentium Pro 200*  60 MB  #3
 Fritz 5.01  Pentium Pro 200*  32 MB  #3
 Junior 4.6  Pentium 200 MMX  32 MB
 Genius 5.0  Pentium 150    #1
 Comet A.75  Pentium 150    #14
 Virtual Chess 1.02  Pentium 150    #10
0  Fritz 5.0  Pentium 150
 Hiarcs 6.0  Pentium 150
* results with a star in the CPU column are by me

Complete Covax suite (3 min/position)
Points (xxx)   Program  CPU  Hash  A (xxx)  B (297)  #1 (58)  #2 (52)  #3 (33)
#4 (52)  #5 (38)  #6 (64)
xxx   Hiarcs 6.0 (engine)  Pentium Pro 200*  32 MB  xxx  169  22  26  18  35  26
 42

* results with a star in the CPU column are by me


--------------------------------------------------------------------------------
Instructions Set up the position(s) and let the program think for exactly 10
minutes (Covax 1), 5 minutes (Covax 2) or 3 minutes (Covax 3), respectively. The
move the program would play at that time is the solution. If it corresponds to
the solution given, you get one point (Covax 1&2) or as many points as indicated
in the suite, including side solutions (Complete Covax), respectively.
(Note that the Complete Covax suite is made up of two parts: Part A is a
selection of the most important positions from Covax 1 & 2, Part B are 6 actual
games taken from CM 5500's "Rate my play" feature.)
In contrast to the Louguet test, it does not matter if the program finds the key
move at 0:01 or at 4:59, as long as it would play it at the end of the time
limit.
Finally sum up all correctly solved positions and you get the final score.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.