Author: Mark Young
Date: 09:52:46 01/04/04
Go up one level in this thread
On January 04, 2004 at 12:40:00, Ricardo Gibert wrote: >On January 04, 2004 at 12:29:15, Mark Young wrote: > >>On January 04, 2004 at 11:46:00, Roger Brown wrote: >> >>>Hello all, >>> >>>I have read numerous posts about the validity - or lack thereof actually - of >>>short matches between and among chess engines. The arguments of those who say >>>that such matches are meaningless (Kurt Utzinger, Christopher Theron, Robert >>>Hyatt et al)typically indicate that well over 200 games are requires to make any >>>sort of statisticdal statement that engine X is better than engine Y. >>> >>>I concede this point. >> >>If you concede this point you don't understand. There is no magic number like >>200 or 2000. The score must be considered. Here is an example: >> >>A score of 17 - 3 in a 20 game match has a certainty of over 99% that the winner >>of the match is stronger then the loser. >> >>A 100 game match ending 55 - 45 only has a 81% chance that the winner of the >>match is the stronger program. >> >>A 200 game match ending 106 - 94 only has a 78 % chance that the winner is >>stronger then the loser. > > >Nothing you have said is really correct because you have ignored the significant >effect of draws in a match. I can only say WHAT!! The last time I checked wins count as 1 point, draws count as 1/2 point, and loses count as 0. So I have no clue what is going on in your brain to make such a comment!! In a 20 game match winning with 17 wins and 3 losses 0 draws is equal to winning with 14 wins 0 losses and 6 draws. You win both matches 17 - 3. The results are one in the same. > > >> >> >>> >>>The arguments of the short match exponents typically centre on other >>>chessplaying characteristics of an engine which may also be of interest to a >>>user - tactical excitement, daring, amazing moves, positional considerations, >>>human like play etc. >>> >>>I also agree that this camp has a valid perspective. >>> >>>I would like to conduct an experiment but I need to ask a few questions first: >>> >>>(1) Is there a minimum timecontrol that is satistically relevant to games >>>played at classical timecontrols? That was really one of the things I wanted to >>>look at but clearly it requires a pool of such games, consistent hardware, etc. >>> >>>I ask this because the long timecontrol devotees have spare hardware, or at >>>least hardware over which they exercise an enormous amount of discretion as to >>>its use. Not all of us are in that fortunate position. >>> >>>Playing 200 games or more at 60 minutes + (which is still fast chess!) would >>>take me to a place where the light does not shine... >>> >>>I am thinking that there may be a relationship - particularly as the subject is >>>an electronic construct - between long games and short ones. It may not be >>>linear but I cannot believe that it is a coincidence that the long timecontrol >>>GMs are also atop the blitz ratings ladder... >>> >>> >>>(2) What is the statistical minimum of games that I would have to play to be >>>able to make some sort of definitive noise? >>> >>> >>>(3) What is the impact - or theoretical impact - of learning on such a match? >>>My personal bias is that if an author implements learning he should be rewarded >>>for it and it should be turned on at the beginning of the match. This speaks to >>>positional and book learning. >>> >>> >>>(4) I am also biased towards using the engine's particular book(s). The >>>opening knowledge that a human chessplayer has is his/hers. An engine should >>>have its own book with it as it goes into battle. Can someone turn off Ms. >>>Polgar's opening book? No? Then the engine should have its book too.... >>> >>> >>>(5) The games would be played on my single processor CPU. That would mean no >>>pondering *if* I understand Robert Hyatt's reasoning on the matter (which I >>>freely admit may not be the case at all!). >>> >>> >>>(6) Are there any other factors? >>> >>> >>> >>>I really would like a way to prove or disprove the position that: >>> >>>(1) Games at shorter timecontrols are essentially worthless and: >>> >>>(2) That matches of 1000 games are required to make statistical statements. >>> >>> >>>Please feel free to comment BUT what I would really like are some answers to the >>>above questions and/or pointers.... >>> >>> >>> >>>Later.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.