Author: Roy Eassa
Date: 08:28:30 10/15/03
Go up one level in this thread
On October 14, 2003 at 17:04:04, martin fierz wrote: >On October 14, 2003 at 13:08:31, Roy Eassa wrote: > >>On October 14, 2003 at 06:46:39, martin fierz wrote: >> >>>nice tests! shredder vs junior and CT15 vs rebel 12 beta at least show trends of >>>favoring one program at short and another program at long time control. >>> >> >> >>Martin, I always have great respect for your postings but in this instance I >>must differ with your conclusion. The same engine won in every match in each >>set. The margins of victory decreased in 7 of the 9 cases where the time >>control was increased (i.e., almost 78% of the time). >> >>To me, this supports the idea that, IN GENERAL, the better engine will win by >>smaller and smaller margins as the time control increases -- leading to the >>possible conclusion that more games at faster time controls is a very efficient >>way to determine which of two engines is better (but not by how much). >> >>Russell Reagan made this (IMHO excellent) post recently, which the new results >>seem to support: >> >>http://www.talkchess.com/forums/1/message.html?320960 > >i guess it's good that we don't agree all of the time :-) >yes, uri has also already pointed out that an alternative hypothesis to explain >the data is that as time controls get longer, results come closer to a drawn >match. >so we have 3 different assumptions: >1) longer time controls favor some engines compared to others >2) longer time controls reduce differences between engines >3) longer time controls make no difference > >and perhaps we can construct some more. while gerhards data is great i don't >think it allows any conclusion about the above possibilites. i hope he continues >with his experiments. > >i tend to believe #1 because i have seen this behavior in my checkers engine in >two different experiments. first, when comparing it to another engine, where at >fast time controls they are equivalent, but at long time controls mine wins >bigtime. second, because i once made a test between a dumb and a smart version >of my checkers program, with the dumb version having an advantage of 4 (IIRC) >ply search depth over the smart one. at low search depths (i.e. short time >controls) the dumb version would beat the smart version. at high search depths >it was the other way round. i attribute this to the difference between strategy >and tactics: the smart version would play the moves that seemed right, but at >low search depth it would fall for traps all the time. at higher search depth >that somehow was not the case. > >i *believe* this, i'm not saying it's true. the other 2 hypotheses are certainly >worth checking too! > >cheers > martin > >PS re russel's post - i just disagree. playing blitz games is not a good idea to >test engine strength IMO... Good answer. At least we can all agree that we're dealing in theories and that our beliefs may be disproven. That's a lot compared to most Internet discussions (including some I have only recently abandoned).
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.