Computer Chess Club Archives


Search

Terms

Messages

Subject: a few games mean almost nothing

Author: Bill McGaugh

Date: 11:59:26 08/24/01

Go up one level in this thread


I have sympathy for Christophe in his attempt to explain the problem...

A few games mean almost nothing...

An example...chessmaster personality mg1 versus the default chessmaster
at game in 5 on a p3-933:

the first 100 game set:  49 to 51...in favor of chessmaster default...
so...is chessmaster better?

the next: 51 to 49...in favor of mg1...
the next: 57.5 to 42.5
then:  47-53
then:  53.5 to 46.5
then:  55 to 45
then:  54 to 46
then:  58.5 to  41.5
then: 59.5 to 46.5
then:  50.5 to 49.5

the totals:  529.5 to 470.5

if I would have accepted the first 100 game set as the "truth", I would have
concluded that mg1 is worse than cm...and imagine, if I had the time to go
through all the pgns...I am sure there are some long strings of wins in a row
for both sides...if any of those subsets were the one that a person viewed
first, they would have been completely deluded about which program version was
stronger...

how can people possibly view a 9 game tournament as anything
definitive...especially people that have played many computer chess games...
among the high level programs, it is close to a coin flip...and a lot of flips
are necessary to figure which side of the coin it is biased toward...


addendum:  1000 games mg2 vs. chessmaster default..game in 5...

55.5-44.5
62.5-37.5
61-39
55.5-44.5
59-41
53-47
57-43
52.5-47.5
53.5-46.5
58-42

total: 567.5-433.5





This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.