Author: Albert Silver
Date: 07:58:48 10/09/04
Go up one level in this thread
On October 09, 2004 at 08:43:52, ThatsIt wrote:
>Hi Albert !
>
>...
>>No, of course not. My main blitz test opponents are Ruffian and List, and my
>>main slower time control opponents are Junior 8 and Hiarcs 9. Mind you, I know
>>and agree that one needs a LOT more games before drawing serious conclusions,
>>but that is extremely time consuming, so before such an investment, one must
>>also use one's judgment. I've already cut short matches where the settings were
>>actually ahead (at least in blitz), because of the way it was playing.
>...
>
>sounds logical but is not really a good idea !
>I've also made some tests with this thought(s)
>i.e.
>
>Pro Deo (another style)
>played with 40moves/1min + 40moves/1min
>vs Shredder 8 1.5 - 38.5 !!
>vs Fritz 8.0.0.26 2.0 - 38.0 !!
>played with 40moves/2min + 40moves/2min
>vs Shredder 8 3.5 - 36.5 !!
>vs Fritz 8.0.0.26 3.0 - 37.0 !!
>played with 40moves/5min + 40moves/5min
>vs Shredder 8 7.5 - 32.5
>vs Fritz 8.0.0.26 6.5 - 33.5
>BUT :-)
>
>played with 40moves/20min + 40moves/20min
>vs Shredder 8 16.5 - 23.5 !!
>vs Fritz 8.0.0.26 12.0 - 28.0 !
>
>A N D
>
>Pro Deo (again another style)
>played with 40moves/1min + 40moves/1min
>vs Shredder 8 12.0 - 28.0
>vs Fritz 8.0.0.26 9.5 - 30.5
>played with 40moves/2min + 40moves/2min
>vs Shredder 8 14.5 - 25.5 !
>vs Fritz 8.0.0.26 11.5 - 28.5
>played with 40moves/5min + 40moves/5min
>vs Shredder 8 12.5 - 27.5
>vs Fritz 8.0.0.26 14.5 - 25.5 !
>BUT :-(
>
>played with 40moves/20min + 40moves/20min
>vs Shredder 8 8.5 - 31.5
>vs Fritz 8.0.0.26 10.0 - 30.0
>
>I think you know what i will tell you ?!
>
>i.e.
>DP don't use the fully given Hash in blitz and/or bullet
>(60MB given - only 28MB use)
>
>maybe some code also isn't used on this short timecontrolls?
>perhabs the other ones do similar?
>
>Best G.S.
I understand, but I think you missed the point of what I said. First of all, I
do not test in bullet since I see no interest in this to be honest. Bullet is
fine of your only interest is in bullet results or seeing tons of games.
However, to judge an engine it is useless. I use the blitz to not only get
results but see what it does. In other words, even if one setting scored 2
points less than another in some blitz match, but played better, I keep that
setting. If I depended entirely on results to make adjustments and test, I would
get nowhere since it would take too much time to find anything meaningful. I
watch the games it plays (not all, but at least a reasonable portion) and when I
do so, I watch how it plays, what its evaluations are, and what its mainlines
are. Even test suites are quite useful in this regard. Not to just test its
tactical acuity, but what it considers the main candidates, and what the
mainlines it gives are. If the main line or candidates look completely
ridiculous, then even finding a few key moves quickly is not enough. The reason
is simply that those main lines are indicative of how it thinks one should
approach the position, and if there is no combination on the board, they are
what it will play. If I don't do this, then to test any given setting I will
need at least 40 machine hours, and I only have one computer and cannot simply
leave it 24/7 to test settings. I do the time consuming matches at night for the
most part.
It's simply not possible IMHO to test everything completely and methodically
while searching for a good setting. You yourself have only made some very small
changes, and I saw no indication of using the many many parameters that Ed
himself has not implemented because he could not make them work to his advantage
(though they weren't necessarily worse) or hasn't tested properly to give the go
ahead. I think one's judgement is one's number one tool when testing and
experimenting, since the number of combinations and values are far too many
otherwise.
Albert
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.