Author: Heiko Mikala
Date: 10:04:49 09/02/99
Go up one level in this thread
On September 01, 1999 at 23:28:18, Dave Gomboc wrote:
>On September 01, 1999 at 19:01:32, Mark Young wrote:
>
>>On September 01, 1999 at 18:26:48, Heiko Mikala wrote:
>>
>[snip]
>>>game/60', 2 * Cyrix/IBM 6x86MX PR-300
>>>
>>> 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7
>>>1 Hiarcs 7.32 ½ 0 ½ 1 ½ 1 0 1 ½ 1 1 1 1 1 1 1 1 13.0/17
>>>2 CSTal II ½ 1 ½ 0 ½ 0 1 0 ½ 0 0 0 0 0 0 0 0 4.0/17
>>
[snip]
>>>And - as always - the same old comment: 17 games are by far not enough to say
>>>which program is stronger. You need at least 60-80 games for that!
>
>With a score this lopsided, 17 games won't be "far not enough".
Sure?!?
Here are some results of test matches, that my own chess engine played against
another well known engine. Note, that I didn't explicitly search for good
examples but simply picked a few matches at random:
1.)
1234567890123456789012345678901234567890
HMChess 1100½10001½1101101½½000½100½111101111001 22.0/40
XXX 0011½01110½0010010½½111½011½000010000110 18.0/40
Look at games 39 to 47. If you only take these games, the score for HMChess
would be 8-1, a very decisive win. But the real result of this match was only a
22-18 ...
2.)
123456789012345678901234567890123456
HMChess 11½100100111111001½01½1001½11½001111 -->
XXX 00½011011000000110½10½0110½00½110000
789012345678901234567890123456789012
--> 0½0101½1000011½11½1½1½111½0000½00100 39.5/73
1½1010½0111100½00½0½0½000½1111½11011 32.5/73
Take games 10 to 15 for example. 6 wins in a row, but a fairly equal end result
for the match.
The first 15 games of this match look very similar to my posted results for the
match CSTal II - Hiarcs, don't they?
3.)
123456789012345678901234567890123456
HMChess 011111100011110110½0101½1100111½½100 -->
XXX 100000011100001001½1010½0011000½½011
789012345678901234567890123456789012
--> 0½01½½101100½0½0½001½10001110011½0½1 38.5/73
1½10½½010011½1½1½110½01110001100½1½0 33.5/73
Take games 2 to 7 for an example of 6 wins in a row.
Endresult only 38.5-33.5 ...
4.)
123456789012345678901234567890123456
HMChess 10110110110½011100100000110101001000 -->
XXX 01001001001½100011011111001010110111
789012345678901234567890123456789012
--> 1001011½½00110½111½10½100111½0111100 37.5/72
0110100½½11001½000½01½011000½1000011 34.5/72
And this is an example, where it was the other way round (maybe the best example
for what I want to say):
If you only take the games 27 to 34 you will find a result of 7-1 for the other
engine. But in the end, HMChess won the match instead, with a score of 37.5-34.5
!!
--------------------------------------------------------------------------
So, I repeat myself and state again: 17 games ar by far not enough to draw *any*
conclusions. Although, as I said, I have the *feeling* that Hiarcs would win a
longer match.
By the way, I originally didn't join this thread to defend CSTal. I'm not
affiliated to the project or the programmers in any way, I simply happen to own
the program and I like it's style.
And yes (this is for Mark), I really tend to be a bean counter in normal life.
My goal is to make my own engine as strong as possible, so I have to be. And
sometimes I would like to know which commercial program is strongest. I'm crazy
sometimes. ;-)
Greetings,
Heiko.
This page took 0.03 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.