Author: Mark Young

Date: 09:29:15 01/04/04

Go up one level in this thread

On January 04, 2004 at 11:46:00, Roger Brown wrote: >Hello all, > >I have read numerous posts about the validity - or lack thereof actually - of >short matches between and among chess engines. The arguments of those who say >that such matches are meaningless (Kurt Utzinger, Christopher Theron, Robert >Hyatt et al)typically indicate that well over 200 games are requires to make any >sort of statisticdal statement that engine X is better than engine Y. > >I concede this point. If you concede this point you don't understand. There is no magic number like 200 or 2000. The score must be considered. Here is an example: A score of 17 - 3 in a 20 game match has a certainty of over 99% that the winner of the match is stronger then the loser. A 100 game match ending 55 - 45 only has a 81% chance that the winner of the match is the stronger program. A 200 game match ending 106 - 94 only has a 78 % chance that the winner is stronger then the loser. > >The arguments of the short match exponents typically centre on other >chessplaying characteristics of an engine which may also be of interest to a >user - tactical excitement, daring, amazing moves, positional considerations, >human like play etc. > >I also agree that this camp has a valid perspective. > >I would like to conduct an experiment but I need to ask a few questions first: > >(1) Is there a minimum timecontrol that is satistically relevant to games >played at classical timecontrols? That was really one of the things I wanted to >look at but clearly it requires a pool of such games, consistent hardware, etc. > >I ask this because the long timecontrol devotees have spare hardware, or at >least hardware over which they exercise an enormous amount of discretion as to >its use. Not all of us are in that fortunate position. > >Playing 200 games or more at 60 minutes + (which is still fast chess!) would >take me to a place where the light does not shine... > >I am thinking that there may be a relationship - particularly as the subject is >an electronic construct - between long games and short ones. It may not be >linear but I cannot believe that it is a coincidence that the long timecontrol >GMs are also atop the blitz ratings ladder... > > >(2) What is the statistical minimum of games that I would have to play to be >able to make some sort of definitive noise? > > >(3) What is the impact - or theoretical impact - of learning on such a match? >My personal bias is that if an author implements learning he should be rewarded >for it and it should be turned on at the beginning of the match. This speaks to >positional and book learning. > > >(4) I am also biased towards using the engine's particular book(s). The >opening knowledge that a human chessplayer has is his/hers. An engine should >have its own book with it as it goes into battle. Can someone turn off Ms. >Polgar's opening book? No? Then the engine should have its book too.... > > >(5) The games would be played on my single processor CPU. That would mean no >pondering *if* I understand Robert Hyatt's reasoning on the matter (which I >freely admit may not be the case at all!). > > >(6) Are there any other factors? > > > >I really would like a way to prove or disprove the position that: > >(1) Games at shorter timecontrols are essentially worthless and: > >(2) That matches of 1000 games are required to make statistical statements. > > >Please feel free to comment BUT what I would really like are some answers to the >above questions and/or pointers.... > > > >Later.

- Re: A question about statistics...
**Roger Brown***09:48:52 01/04/04*- Re: A question about statistics...
**Mark Young***10:14:22 01/04/04*- Re: A question about statistics...
**Roger Brown***10:31:50 01/04/04*

- Re: A question about statistics...

- Re: A question about statistics...
- Re: A question about statistics...
**Ricardo Gibert***09:40:00 01/04/04*- Re: A question about statistics...
**Mike Byrne***10:01:59 01/04/04*- Re: A question about statistics...
**Ricardo Gibert***10:56:30 01/04/04*

- Re: A question about statistics...
- Re: A question about statistics...
**Mark Young***09:52:46 01/04/04*- Re: A question about statistics...
**Ricardo Gibert***10:56:03 01/04/04*

- Re: A question about statistics...
- Re: A question about statistics...
**Peter Berger***09:47:25 01/04/04*- Re: A question about statistics...
**Ricardo Gibert***10:46:48 01/04/04*- Re: A question about statistics...
**Dieter Buerssner***14:44:08 01/04/04*- Re: A question about statistics...
**Peter Fendrich***17:55:33 01/04/04*- Re: A question about statistics...
**Dieter Buerssner***01:56:23 01/05/04*- Re: A question about statistics...
**Dieter Buerssner***04:55:26 01/05/04*- Re: A question about statistics...
**Peter Fendrich***15:11:25 01/05/04*- Re: A question about statistics...
**Dieter Buerssner***11:03:05 01/06/04*- Re: A question about statistics...
**Peter Fendrich***16:03:58 01/06/04*- Re: A question about statistics...
**Dieter Buerssner***09:23:00 01/07/04*- Re: A question about statistics...
**Peter Fendrich***05:53:58 01/09/04*- Re: A question about statistics...
**Dieter Buerssner***10:26:17 01/09/04*- Re: A question about statistics...
**Peter Fendrich***05:05:34 01/12/04*

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...
- Re: A question about statistics...
**Ricardo Gibert***16:46:51 01/04/04*- Re: A question about statistics...
**Peter Berger***16:51:15 01/04/04*- Re: A question about statistics...
**Ricardo Gibert***16:56:12 01/04/04*

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...
- Re: A question about statistics...
**Bo Persson***14:15:33 01/04/04*- Re: A question about statistics...
**Ricardo Gibert***16:38:53 01/04/04*- Re: A question about statistics...
**Mike Byrne***18:04:53 01/04/04*

- Re: A question about statistics...

- Re: A question about statistics...
- Re: A question about statistics...
**Peter Berger***13:55:07 01/04/04* - Re: A question about statistics...
**Mike Byrne***11:57:59 01/04/04*- Re: A question about statistics...
**Ricardo Gibert***16:22:43 01/04/04*- Re: A question about statistics...
**Sandro Necchi***01:13:21 01/06/04*- Re: A question about statistics...
**Sandro Necchi***23:07:08 01/06/04*- Re: A question about statistics...
**Rolf Tueschen***04:32:05 01/07/04*

- Re: A question about statistics...
- Re: A question about statistics... To SN
**Rolf Tueschen***06:18:57 01/06/04*

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

- Re: A question about statistics...

This page took 0.1 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.