Author: Ratko V Tomic
Date: 14:01:34 09/28/99
Go up one level in this thread
>>Well, yes, the fewer games the greater uncertainty. My point was: how do you >>distribute the uncertainty fairly or optimally? Do you take one company and >>reduce its uncertainty (by sticking with N>100) and simultaneously greatly >> widen the uncertainties for other companies/authors (since they get N=0 >> games on 450, thus it takes extrapolation from 200 to approximate the >> correct program rank)? >> >>Of course not. The overall uncertainty (sum of squares of individual >>uncertainties) is minimized by making all the individual uncertainties >> equal to each other. So mathematically their "optimization" method of >> uncertainties is ridiculous. >> >If you really did understand the math (which you obviously *do not*) then you >would *know* that the top entries are mathematical peers. They post the >standard deviation as well, and all of the top entries are within one standard >deviation of each other. I think you missed the point since your reply actually supports my argument. Namely, since the top programs (including the non-CB ones) are within the statistical uncertainties from each other, that makes the SSDF decision to preordain the 4 CB programs to be on the top of the list, before the test even started, even less justifiable. My earlier point was that it wasn't justifiable mathematically either, to give only the CB programs fast machines, since that doesn't improve overall uncertaianty (as you tried to argue with by bringing up the 100 game rule they have), it makes it worse. Another reason was a blatant unfairness, which is quite obvious if you take the example I gave with a car race, where we can all be more detached when judging. As SSDF acknowledges (on their web page) it was aware that using uneven hardware for CB only was unfair and rejected such proposal from CB. > What is the alternative to the CB autoplayer? None if each side sticks with its position. CB will play only using their proprietary autoplayer (which some others do not trust). The proper alternative to CB's attempt at strong-arming the SSDF whould have been to skip CB from the main list and explain that CB refused to be tested on equal grounds with others. Had SSDF stood its ground, they would have not alienated others and CB would have been forced to come around. According to SSDF web page, they apparently tried to reason with CB, but to no avail, so they bent the integrity of their own procedures instead and lost the others (and some of their credibility) instead.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.