Author: martin fierz
Date: 14:04:04 10/14/03
Go up one level in this thread
On October 14, 2003 at 13:08:31, Roy Eassa wrote: >On October 14, 2003 at 06:46:39, martin fierz wrote: > >>nice tests! shredder vs junior and CT15 vs rebel 12 beta at least show trends of >>favoring one program at short and another program at long time control. >> > > >Martin, I always have great respect for your postings but in this instance I >must differ with your conclusion. The same engine won in every match in each >set. The margins of victory decreased in 7 of the 9 cases where the time >control was increased (i.e., almost 78% of the time). > >To me, this supports the idea that, IN GENERAL, the better engine will win by >smaller and smaller margins as the time control increases -- leading to the >possible conclusion that more games at faster time controls is a very efficient >way to determine which of two engines is better (but not by how much). > >Russell Reagan made this (IMHO excellent) post recently, which the new results >seem to support: > >http://www.talkchess.com/forums/1/message.html?320960 i guess it's good that we don't agree all of the time :-) yes, uri has also already pointed out that an alternative hypothesis to explain the data is that as time controls get longer, results come closer to a drawn match. so we have 3 different assumptions: 1) longer time controls favor some engines compared to others 2) longer time controls reduce differences between engines 3) longer time controls make no difference and perhaps we can construct some more. while gerhards data is great i don't think it allows any conclusion about the above possibilites. i hope he continues with his experiments. i tend to believe #1 because i have seen this behavior in my checkers engine in two different experiments. first, when comparing it to another engine, where at fast time controls they are equivalent, but at long time controls mine wins bigtime. second, because i once made a test between a dumb and a smart version of my checkers program, with the dumb version having an advantage of 4 (IIRC) ply search depth over the smart one. at low search depths (i.e. short time controls) the dumb version would beat the smart version. at high search depths it was the other way round. i attribute this to the difference between strategy and tactics: the smart version would play the moves that seemed right, but at low search depth it would fall for traps all the time. at higher search depth that somehow was not the case. i *believe* this, i'm not saying it's true. the other 2 hypotheses are certainly worth checking too! cheers martin PS re russel's post - i just disagree. playing blitz games is not a good idea to test engine strength IMO...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.