Author: Bob Durrett
Date: 09:59:41 01/23/04
Go up one level in this thread
On January 23, 2004 at 12:16:55, Rolf Tueschen wrote: >On January 23, 2004 at 12:10:08, Richard Pijl wrote: > >>>So from the match results you can conclude with 95% certainty that >>>Shredder 8 is at least one rating point better. >> >>Just to be absolutely clear on this: >>This is a result from matchplay between two engines. The difference calculated >>is of course only valid for results between those two engines. In real life it >>may not mean much. Shredder 8 may turn out to be weaker or much stronger when >>playing other engines. That's just the limitation of the model and measurements >>used. > >This is all correct, Richard, but that is NOT what I had to criticise. The point >was that the result mentioned was NOT enough to conclude a better strength. >Please also do read the other postings I wrote on the topic today. And I can >assure you that I dont play around and make jokes. > >Rolf > > >> >>Richard. In the "real world" of the chess program USER, there often is the question: "Which engine shall I use today if I wish to have the very best chance of getting correct analysis?" If the user seeks guidance from the esoteric "world of the chess program testers," then a choice must be made based on available information as presented by those "testers." This may put the unfortunate user "between a rock and a hard place" because the testing guys may have done a poor job of presenting their findings, even if they did not err in their [statistical] analyses. If I were the user and the data presented to me indicated that engine #1 was slightly better than engine #2 I would go with engine #1 even if the confidence level was said to be extremely poor. The issues about whether engine #1 is one point better or 100 would be irrelevant. Bob D.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.