Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: A question about statistics...

Author: Russell Reagan

Date: 09:38:32 01/05/04

Go up one level in this thread


On January 04, 2004 at 12:07:19, Robert Hyatt wrote:

>I think you need to do something in 60 minutes at least, plus some sort of
>secondary time control or increment.

What hardware do you have in mind? If the hardware is twice as fast as what you
have in mind, then is G/30 acceptable? Or if the hardware is twice as slow, do
you now need G/120?

Ten years ago there was a time control that was acceptable to decide the world
champion of computer chess. Now hardware is much faster (100x?), but a time
control 100x faster is thought to be silly by most people.

I don't think we need super long time controls at all to decide which engine is
better. I think longer time controls hide the weaknesses of the weaker engine,
so the games become less meaningful as to which engine is stronger (not to
mention that the games take a heck of a lot longer).

Take two extreme examples. At an infinite time control, all engines will play
perfectly. At G/1, even the slightest weakness can become glaring, and fatal.

I think the optimal setting should be long enough to cover up weaknesses to some
degree (ex. one engine shouldn't lose every game on time), but not so long that
the weaker engine's weaknesses are hidden totally.

I have seen it both ways in matches I have run, and also from results that other
people post here. In very fast games at G/1, I've seen a strong engine like SOS
lose almost every game on time (subpar time management). At a longer time
control like 40moves/4hrs, SOS was only about a game or two down against Ruffian
after 40+ games. How often do you see someone post a result of some engine
scoring about even with a top program, or see someone posting, "Engine X is very
strong at longer time controls."? I remember seeing these kinds of posts quite a
bit.

So if the goal is to determine which engine is better, a sufficient shorter time
control is best I think. That way you get a lot more games played, and the
weaknesses of the weaker engine are not masked. What do you think?



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.