Author: James B. Shearer
Date: 19:43:03 09/19/98
Go up one level in this thread
On September 18, 1998 at 16:35:25, Thorsten Czub wrote:
>On September 18, 1998 at 04:09:43, James B. Shearer wrote:
>
>> Well consider the fact that the only game mclane chose to replay was a game
>>in which fritz beat rebel. This is a subjective decision and one might suspect
>>the game would not have been replayed had rebel won.
>
>but your consideration is nonsense.
>I did not "replay" it because rebel lost, i replayed it because ed complained at
>me that not the right book was used.
But would Ed have complained if Rebel had won? Suppose (ignoring draws
for simplicity) Rebel had a 40% chance of winning with the wrong book and a 60%
chance of winning with the right book and that Ed would not have complained if
Rebel had won with the wrong book. Then it is easy to see that your procedure
has raised Rebel's winning chance from the .60, it was entitled to, to .76.
Also the wrong book was used for several games and you only replayed one.
In any case my point is not that your decision was right or wrong, but
that it had a subjective element because there is no obvious right thing to do.
Similarly for your current problem about what to do about chessmaster crashing.
These subjective decisions introduce the possiblity of bias in the results.
This is why scientists generally try to specify the complete experimental
protocal in advance including how to handle errors and other unanticipated
events.
>
>If any other participant would have complained for whatever reason, and it would
>have been reasonable, i would have done the same for any other program too.
>But the complain by ed is the only complain so far...
>
>> Similar plausible decisions to throw out some data or change the protocal
>>in mid experiment have led to invalid results many times.
>> James B. Shearer
>
>I can only laugh about your statements.
No one is so blind as he who will not see.
James B. Shearer
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.