Author: Pallav Nawani
Date: 04:04:21 07/17/05
Go up one level in this thread
On July 17, 2005 at 05:22:47, Sandro Necchi wrote: >I have been laughing a lot (maybe crying on the ignorance would have been more >appropriate?)reading many wrong statements about testing and Elo lists. I guess you know better than us, but your arguments hold no water. >so, for those who are new and do not know, SSDF list is the best for the >following reasons: > >1. They use 2 computers and the program complete with own book and ETG, with >own gui and best setting as suggested by the programmer. Depends on what you want to test. I you want to understand the strength of the _complete package_, then yes, using own Gui, using own Book with learning on is the best way. However, testing the strength of the _engine_ in isolation also means using the openings where it might not play as well. This is also a valid way of testing, despite claims to the contrary. >2. They use long time controls (40/2h 20/1h; international level) only. Irrelevant. For _rating_ (Mind you, _ratings_) Any time control is good enough as long as it is not so small that programs lose on time. CEGT time control is long enough, IMHO. What is important is having enough number of games. SSDF do have a good number of games, of course, but just not enough to differentiate between two programs that are very close in strength. >3. They use the same hardware for all programs. I agree that this is a good thing, because this makes the matches more consistent. On the other hand, different hardware affects different programs differently, so (in theory) the result we have is only correct for SSDF harware. For instance, how would crafty perform if they used 64 bit Opterons? However, the differences are usually small. >4. They use a very wide range of programs and not only the new ones to get more >reliable results. I think every rating list does that. >5. Ponder on and learning are activated. For rating purposes, ponder on is irrelevant, since pondering is effectively nothing more than giving more time to a engine. For learning see the answer to your point 1. >So, anybody can test in a different way as they wish, but to claim that system >is better or replacing the SSDF system is pure nonsense! While I agree that SSDF gives us a good idea of the complete package, claiming that SSDF is better is also pure nonsense. For instance, Shredder 7.04 and Shredder 8 are very close in the SSDF list. Do you think that Shredder 7.04 and Shredder 8 have the same strength? Every rating list has its anomalies.. Best Regards, Pallav
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.