Author: Axel Schumacher
Date: 23:04:48 08/07/03
Hi all,
I finally had the opportunity to convert the 84.000 games database from my
tourney at: http://www.grailmaster.com/misc/chess/comp/compindex.html into a
pgn-file. I calculated the Elo with EloStat and found, that the Elo-ranking was
quite different than that with Fritz (on the homepage is still only the Fitz
calculated version). Details of the tourney on the webpage.
Not only the absolute values are different, but also the ranking. The biggest
difference in the top of the table (over 790 engine versions) can be seen with
Chess Tiger, which EloStat doesn't seem to like :-)
Here the Top 16 with EloStat (only best ranked version of each engine):
Program Elo + - Games Score Av.Op.
Draws
1 Deep Fritz 7 : 2742 24 26 566 61.3 % 2662 27.2 %
2 CM9000 M2v.5 : 2714 34 31 338 57.2 % 2663 29.3 %
3 Chess Tiger 14.0 : 2688 11 14 2462 64.6 % 2583 25.7 %
4 Hiarcs 8 : 2670 17 16 1264 56.6 % 2625 26.3 %
5 Shredder 7.00 : 2663 29 24 558 51.9 % 2650
24.6 %
6 Junior 8.0.0.2 : 2662 44 36 234 52.6 % 2644 25.6 %
7 Ruffian 1.0.1 : 2659 16 16 1399 58.7 % 2598 27.7 %
8 Rebel 12b beta9 : 2625 56 55 116 60.3 % 2552 29.3 %
9 Deep Sjeng 1.5 : 2613 36 36 279 59.0 % 2550 26.9 %
10 Aristarch 4.21 : 2607 38 36 273 55.9 % 2566 23.1
%
11 SOS.3 for Arena : 2601 27 31 449 60.9 % 2523 23.8
%
12 Crafty 19.04 Stein : 2599 56 50 123 58.1 % 2542 31.7
%
13 Delfi 4.2 : 2599 42 36 237 54.6 % 2566 28.3
%
14 List 504 : 2580 27 24 586 53.2 % 2557 21.8
%
15 Smarthink 0.16b2 : 2579 44 41 204 57.1 % 2529 27.0
%
16 Gandalf 4.32UCI : 2573 26 25 577 56.8 % 2525 23.6
%
The same calculation by the Fritz 8-Gui:
1. Deep Fritz 7 2755 562
2. Chess Tiger 14.0 2731 2462
3. CM9000 M2v.5 2725 338
4. Hiarcs 8 2699 1260
5. Ruffian 1.0.1 2692 1397
6. Junior 8.0.0.2 2681 260
7. Shredder 7.00 2680 558
8. Rebel 12b beta9 2660 111
9. Deep Sjeng 1.5 2653 273
10. SOS.3 for Arena 2651 447
11. Aristarch 4.21 2646 269
12. Crafty 19.04 Stein 2645 118
13. Delfi 4.2 2635 233
14. Smarthink 0.16b2 2629 200
15. Gandalf 4.32UCI 2626 577
16. List 504 2623 582
Quite some interesting differences... eh?
Another problem is still present, even with EloStat (Startvalue 2400). The lower
ranked engines are get still a much too high Elo. How to avoid that? I saw
several other lists, where the list-authors made it possible that the Elo-values
are lower. If I start with a lower start-value of e.g. 2000, than the top
engines are ranked far too low. Any suggestions?
Example:
EloStat, some of the lower ranked engines:
728 Yawce 0.16 : 1962 58 33 263 31.9 % 2094 6.8
%
733 Raffaela : 1951 78 46 130 30.0 % 2098 12.3
%
736 Nero 5.3 : 1934 114 60 81 30.2 % 2079 1.2
%
755 Pierre 1.7 : 1861 60 30 290 30.2 % 2007 3.8
%
773 ROBOKewlper 0.047 : 1778 143 55 71 15.5 % 2073 14.1
%
775 Bigbook 3.1 : 1765 48 24 443 28.0 % 1929 9.5
%
781 König Schwarz : 1717 53 42 182 36.0 % 1817 20.3
%
787 Kace 0.8 : 1643 123 75 47 22.3 % 1860 23.4
%
and the same with Fritz (even much higher values):
Yawce 0.16 2080 262
Nero 5.3 2073 79
Raffaela 2064 130
Pierre 1.7 1983 288
Bigbook 3.1 1902 441
ROBOKewlper 0.047 1898 69
König Schwarz 1880 182
Kace 0.8 1805 47
Axel
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.