Author: Robert Hyatt
Date: 09:50:39 12/17/03
Go up one level in this thread
On December 17, 2003 at 12:38:45, Daniel Clausen wrote:
>On December 17, 2003 at 09:35:16, Robert Hyatt wrote:
>
>[snip]
>
>>And, as I suggested previously, if, after a program leaves book, it is
>>in an obviously won or lost position, the game gets aborted and the next
>>one started. There is no place for "book kills" when the goal is a time
>>handicap match.
>
>In order to reduce threads like 'this opening position is lost! no it's not! yes
>it is! it's lost when you use bitboards! but fisher would win this position vs
>DB!' it would be good to 'formalize' won/lost positions after the opening.
>
>You could declare an opening won/lost if one of the engines evaluates its first
>move out of book with a score outside a predefined score-window [X, Y]. ([-0.5,
>+0.5] could be an example) Some points:
>
>- I intentionally used two variables so it's possible to have an assymetric
>window (no clue whether that could be helpful or not)
>- You don't capture positions where a human being with comp-chess knowledge
>knows, that one engine _will_ lose but the scores of the engines won't catch it
>- the scores for this score-window have to be adjusted (+1 should mean approx 1
>pawn advantage)
>- everything else I forgot :)
>
>Ideas? Comments? Shrieking epitaphs?
>
>Sargon
>
>PS. It's funny - we often claim that "{small number} games are not enough!" but
>now we still make this experiment ;)
It depends on what you want to prove.
If you want to show exactly how much better one program is than another, then
the more games, the merrier. But if you simply want to show that a handicap
is significant, a few games can do that, assuming that they don't all end in
draws.
For example, in the current Rebel vs Crafty odds match, I think it pretty
clear that the time handicap is _very_ significant. 4 wins vs 1 loss and
2 draws is pretty convincing, since the games are slow time control to start
with. The question really isn't "how much better is the handicapped side"
but "is the handicapped side better?"
Speed is not everything, but it is very important.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.