Author: Melvin S. Schwartz
Date: 08:45:42 05/30/99
Go up one level in this thread
On May 30, 1999 at 00:59:56, James Robertson wrote: >On May 29, 1999 at 23:49:47, Melvin S. Schwartz wrote: > >>What do you think Mr. Hyatt means when he says: >> >>"If program A on hardware B beats program D on Hardware E - does that say much >>about A compared to B? This belies the principles of science - you have to have >>a uniform platform for all participants to make any kind of judgement". > >Yes.... if out of the blue you play program A on hardware B vs program D on >hardware E you have no clue how the program's relate to each other. > >But..... >Say, hypothetically, that program AChess is rated 2000 on a P90. > >Then suppose AChess on a P90 plays AChess on a P200, and loses by 60%. That >would put AChess on a P200 rated about 2100. > >Now we have some concrete evidence; AChess on a P90 is 2000, AChess on a P200 is >2100. > >Now suppose we play AChess on a P200 against BChess on a P90, and AChess wins by >60%. That puts BChess' rating at 2000. > >We have discovered BChess on a P90's rating, and we have used uneven hardware. >We will reliably be able to predict a ~50-50 score vs AChess on a P90 vs BChess >on a P90. > >Do you see how this works? Uneven hardware does not compare programs, it >compares program's on a certain hardware. > >James Hello James I don't like conclusions drawn from hypothetical theory when testing in reality is available. I have not seen it stated by SSDF that they are testing as you suggest. If they are testing two software programs on two different processors and then reversing the process, I will accept that as a practical soution given the fact they haven't the resources to do it otherwise. The post I responded to only indicated testing of Hiarcs 6 on a P90 versus Nimzo 98 on a P200MMX with the score 0-1. If they are going to repeat the match reversing the processors, then the reply I received should have stated this. The reply I received merely said this is the way we do it and will continue to do it this way regardless of what you say! Regards Mel> > >> >>Now, I understand that to mean exactly what I have said over and over again. And >>that is simply the fact you have to have chess programs running on the same type >>of hardware to make a proper judgement. Now, isn't that what Mr. Hyatt is >>saying? Or perhaps you feel this is the first time he is wrong and you are >>right? >> >>Mel
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.