Author: Harald Faber
Date: 01:35:53 09/02/99
Go up one level in this thread
On September 01, 1999 at 23:40:33, Dave Gomboc wrote: >>>>>>AMD-K6-200, most out of 64MB RAM, 3min/move. >>> >>>>EXACTLY 3min/move EACH. >>> >>>What is the sense to give them 3 moves each exactly ??? >>>CSTal plays weaker in this time-control than in 40/120. >>>so you weaken it. >> >>Yes, but the other programs had the EXACT same limitation. Obviously, if this >>configuration weakens CSTal (as it should), it will also weaken it's competitors >>(as it should). The test is still valid. It's just a different type of test AND >>people have to take it for what it is and not try to make any other assumptions >>about it. One cannot assume from a test like this that CSTal is weaker than >>these particular opponents if permanent brain is turn on and/or the program >>decides when how much time to spend on each move (which I believe is an >>assumption Harald has made) because it lost games where these features were >>turned off. > >Time management has become relatively important. I'm not sure how much penalty >there is to using a naive time management implementation than a sophisticated >one. It would be interesting to hear the opinion of commercial developers on >this. My guess would be 20+ elo. It has been suggested that much work has gone >into CSTal's time management code, so I sympathize with Thorsten's point of view >here. > >I think it's best to test with a time control of n moves in x minutes, on two >machines, with pondering on, where n should be significantly more than 1. :-) Sure. But it doesn't change or lower the fact that BOTH PROGRAMS HAD THE SAME CONDITIONS. PERIOD. >>>>>Also : played on ONE machine or played on 2 machines ?? >>>>>permanent brain on/off ? >>> >>>>2 machines pb=off as they were played via e-mail. >>> >>>pah - than the games are nonsense ! you do not test the programs, >>>you test something different. >> >>The games are not nonsense (you like that word I guess). Neither are the tests. >>If you test G5 in your "prefered" configuration of permanent brains on and >>program decided time per move, it would still be a valid test of JUST THAT >>configuration (and no other configuration). It wouldn't tell you how well CSTal >>or any other program would probably due in standard game times. >> >>What is it with this notion that testing MUST be done under x y z conditions and >>cannot be done under p q r conditions? Results of any testing must take into >>account the conditions set up, but that does not mean that a non-standard test >>has no validity. Granted, if the two programs had different conditions, then >>your point would be valid. But, the conditions were the same for both opponents. >> >>What tests like these CAN show (if enough games are played) is that there could >>be configurations that are good for one program, but bad for another program. >>But, that does not mean that the games are nonsense. >> >>KarinsDad :) > >It doesn't mean that the games are representative of the results one would get >when playing comp-comp under tournament conditions, either. Of course, you know >that already. >Dave NOONE claimed that!
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.