Author: Dave Gomboc
Date: 20:40:33 09/01/99
Go up one level in this thread
On September 01, 1999 at 17:32:41, KarinsDad wrote: >On September 01, 1999 at 16:45:45, Thorsten Czub wrote: > >>On September 01, 1999 at 09:32:38, Harald Faber wrote: >> >>>>>AMD-K6-200, most out of 64MB RAM, 3min/move. >>>> >>>>???? >>>> >>>>3 min/move ?? 40/120 or average 3 minutes ? >> >>>EXACTLY 3min/move EACH. >> >>What is the sense to give them 3 moves each exactly ??? >>CSTal plays weaker in this time-control than in 40/120. >>so you weaken it. > >Yes, but the other programs had the EXACT same limitation. Obviously, if this >configuration weakens CSTal (as it should), it will also weaken it's competitors >(as it should). The test is still valid. It's just a different type of test AND >people have to take it for what it is and not try to make any other assumptions >about it. One cannot assume from a test like this that CSTal is weaker than >these particular opponents if permanent brain is turn on and/or the program >decides when how much time to spend on each move (which I believe is an >assumption Harald has made) because it lost games where these features were >turned off. Time management has become relatively important. I'm not sure how much penalty there is to using a naive time management implementation than a sophisticated one. It would be interesting to hear the opinion of commercial developers on this. My guess would be 20+ elo. It has been suggested that much work has gone into CSTal's time management code, so I sympathize with Thorsten's point of view here. I think it's best to test with a time control of n moves in x minutes, on two machines, with pondering on, where n should be significantly more than 1. :-) >> >>>>Also : played on ONE machine or played on 2 machines ?? >>>>permanent brain on/off ? >> >>>2 machines pb=off as they were played via e-mail. >> >>pah - than the games are nonsense ! you do not test the programs, >>you test something different. > >The games are not nonsense (you like that word I guess). Neither are the tests. >If you test G5 in your "prefered" configuration of permanent brains on and >program decided time per move, it would still be a valid test of JUST THAT >configuration (and no other configuration). It wouldn't tell you how well CSTal >or any other program would probably due in standard game times. > >What is it with this notion that testing MUST be done under x y z conditions and >cannot be done under p q r conditions? Results of any testing must take into >account the conditions set up, but that does not mean that a non-standard test >has no validity. Granted, if the two programs had different conditions, then >your point would be valid. But, the conditions were the same for both opponents. > >What tests like these CAN show (if enough games are played) is that there could >be configurations that are good for one program, but bad for another program. >But, that does not mean that the games are nonsense. > >KarinsDad :) It doesn't mean that the games are representative of the results one would get when playing comp-comp under tournament conditions, either. Of course, you know that already. Dave
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.