Author: Melvin S. Schwartz
Date: 08:36:17 06/01/99
Go up one level in this thread
On May 31, 1999 at 13:45:40, blass uri wrote: > >On May 31, 1999 at 11:44:15, Melvin S. Schwartz wrote: > >> >>On May 30, 1999 at 16:03:40, Daniel Karlsson wrote: >> >>>On May 30, 1999 at 11:24:22, Melvin S. Schwartz wrote: >>> >>>[Snip] >>>> >>>>The amount of points you speak of is of a hypothetical nature. If you like >>>>comparing software with A at B speed and C at D speed, then we just simply >>>>disagree. >>>> >>>>Regards, >>>>Mel >>> >>>Suppose program A on hardware B gets a 70% score against C on D. Now if E on B >>>gets a 75% score against C on D, wouldn't that be a good indication that E on B >>>is stronger than A on B, i.e. E is stronger than A on the same hardware. >>> >>>Match AB and EB against several opponents, calculate ratings from the scores and >>>you get a pretty good rating list. This is basically what SSDF are doing. >> >>You may get a pretty good idea of a rating but how accurate is it? I suspect we >>are dealing with a strong assumption here. It may be the only way that SSDF can >>do it, but Shep's site is where you'll find tournaments where programs ARE >>competing against each other on EQUAL hardware. Now, I am compelled to believe >>Shep's results with more authenticity than SSDF's method. > >The reason to believe more the ssdf results is because the ssdf are based on >more games relative to Shep's games and every top program has more than 100 >games(usually some hundreds of games). > >The main problem that I have with the ssdf results is that most of the games are >not public so I cannot check if there are mistakes in these results. > >I found in the past mistakes in one match Junior5(p200)-Rebel8(P90) when Junior5 >was slowed down by a significant factor in 4 games because the tester ran >another application in the same time. >The tester repeated the games. > >It is impossible to discover these mistakes when the games are not public. > > > Another problem I have >>with SSDF is their opponents for Chessmaster 6000 have an average rating, if >>memory serves me well, more than 100 points below that of Hiarcs7 or Fritz 5.32 >>just to name a few. In my opinion, and this is just my opinion, I believe they >>should confine their testing to the top programs because that may allow them to >>use the same hardware for all. Also, since we know for example that Hiarcs 7 is >>better than Hiarcs 6, why do they continue to test Hiarcs 6? This applies to >>other outdated programs as well. Do you see industry extensively testing newer >>and better cars, computers, TV's, etc., when it has already been etablished that >>new products from the same maufacturer are superior to their older models? > >Usually new programs are better than old programs but it is not obvious. >There is at least one case when the ssdf results does not show improvement and >Mchess8's rating is worse than Mchess7. Hi Uri I wasn't saying new products defies reasoning to test older models, but rather once a newer model has proven itself superior to the older model - that would indicate further testing of the older model to be of little, if any, significance. I make the above point a major issue because I prefer to see program A tested against program B on the same hardware. It is written in the manuals of both Hiarcs7 and Fritz 5.32 that the speed of the processor affects the stength of the programs. I have no doubts about that. Therefore, I feel that if only the best representatives from each manufacturer were tested - this may allow the use of equal hardware because there wouldn't be a need to test unnecessary programs. I know that SSDF claims that testing on equal hardware isn't necessary to get true ratings, but I do have a problem with that. It is most logical to me that testing be done on equal hardware to eliminate as many variables as possible and get the most accurate rating possible. By the way, I personally would like to thank you for the most even-tempered response to my posting. You are the FIRST person to respond without inflaming the situation. I have put my foot in my mouth in my haste to reply to the many who have answered my posting. I have this character flaw which dictates me to answer all in a timely fashion. This is troublesome for me because I have so many things to do besides keeping up with answers to my posting. I will from now on only reply when I am not pressed for time. Once again, thank you for responding without expressing yourself in an angry fashion. I believe you would make an excellent moderator! I say that not just from this response but from observing your other postings. Your calm and intelligent approach to answering a post is quite refreshing and most appreciated. Kind regards, Mel > >Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.