Author: David Rasmussen
Date: 13:58:11 06/11/01
Go up one level in this thread
On June 11, 2001 at 16:29:53, Gian-Carlo Pascutto wrote: >On June 11, 2001 at 16:08:19, David Rasmussen wrote: > >>If you simply want to know who performed best of two engines, then you are >>right. But you can conclude anything from that. > >Oh yes. My conclusion is that when I match the two >programs again I know with 95% certainty which one is >going to lose in the long run, if either. > That's true. >>Certainly not that one is a better chess player than the other. > >Well, among those 2 engines the winner is the better chess >player. Unless you define 'better chess' player as something >else than that which will win the most games. > I do indeed define it as something else, when it comes to chess programs. The only way, for me at least, to determine which program is best, is to see how it scores against various opponents. That's even the case with humans, IMO. One player can more or less consistently beat another player who more or less consistently beats a third player who more or less consistently beats the first player, because of different styles/strengths. A higher rated player can even have a bad perfomance record against a specific lower rated player, because of differences in styles/strengths. The only thing you can conlude with an experiment as the one you describe, is that within a certain margin of error, program A will beat program B at least x percent of the times. >It doesn't say a thing about performance vs something else, >but I find it interesting to compare different versions of >the same engine. If I fiddle and it suddenly plays significantly >better than the old version, that is a good indication that >that setting is better even vs other programs/humans. > No it isn't. It is wellknown that this type of selftesting leads to inbreeding, because both programs can't punish eachother for mistakes they don't understand.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.