Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Some Basics for Testing Conditions

Author: Rolf Tueschen

Date: 04:28:30 11/02/03

Go up one level in this thread


On November 01, 2003 at 21:31:49, Mike S. wrote:

>On November 01, 2003 at 15:21:42, Rolf Tueschen wrote:
>
>>On November 01, 2003 at 09:30:02, Mike S. wrote:
>>
>>>(...)
>
>>Thanks for the response BTW. But I must tell you that I am not willing to give
>>you some extra lessons in experimental design and stats.
>
>Thanks, I suffer from an overdose of statistics anyway, when reading the message
>boards and computerchess homepages.
>
>>You are in defense for ChessBase.
>
>:-)) If I had told you I bought a cat and wrote a poem about the moon, you'd
>probably reply too, that I did this in defense of ChessBase.
>
>>You want to defend the intended practice of
>>lending the general.ctg from ChessBase to The King.
>
>I'm not so sure that ChessBase is happy about that practise. - In recent
>King/SSDF discussions (also before the test started a few days ago), I've always
>written that my idea would be, to generate a simple but solid "neutral" opening
>tree for that purpose, from a good game collection. Mixing products would not be
>my first idea. But OTOH, this would have meant one more disadvantage for King,
>raising protest from the other side so to speak. In such a simple tree,
>computerchess specific editing (not necessarily engine specific but more
>general) would be missing, in addition to the fact that the King 3.23 tests
>start very late after the relase, which was long ago and competitors have
>released new versions since then. Therefore, I think it's quite ok too, as a
>compromise. It isn't meant as a kind of new rule IMO, but as an exception from
>the existing testing practise, also because the common usage of King (among
>computerchess fans) is so much different from the usual out of the box/default
>philosophy.


The marketing philosophy of ChessBase perhaps goes into the direction of
becomming a sort of state of the art tech center for all the professional
engines and the amateurs too. Like the marketing strategy with their server.

The moment they have the absolute monopol they will start to pick up the money.
But I detest to give you more extra lessons in the economy of monopolism. ;)



>
>>Therefore you rely on the results of Michael42.
>
>I don't especially rely on these results, I took them as an illustration how
>books can make a very small difference only, sometimes. But I don't think these
>results are typical in this extreme form, book vs. no book and such a small
>difference.

But this was exactly what you did imply. Michael42 had seperated the only
variable he wanted to examin. Of course this was a hoax.


>
>>But
>>these results don't mean anything and your relying is biased and typical for a
>>CSS spin doctor in defense of a ChessBase marketing idea.
>
>This is a strange remark. What marketing idea? The general.ctg is not a product
>of it's own, it's an old book sold with Fritz 6 some years ago.

Good enough for weaker amateurs - this is the marketing idea. Because - why
should this be provided for "free" in the long run?? {-see above}

>
>>>He had very good results with a special book he has compiled... I don't see why
>>>this should do worse with learning on. I think it would benefit from the
>>>learning function just like other books do as well. I really think the book
>>>comparison is valid.
>>
>>No, it's not. He makes clowns out of Sandro, Jeroen and other book doctors.
>
>I don't think so!
>
>But books are overrated in this circus. Good, useful, can decide games
>*somtimes*, but in the general opinion much too overrated. The engines rule the
>game, not the opening books.


Again, you are very naive. (quote from Sandro, not me!)

You are a bit too much fixated on the top of the pops. And even there you
overlook the decisive element of books for the title competition. Of course it
is still true that the engine is the most important element, no doubt.


>
>>created his book for whom??? For SHREDDER or both? But if he created his book
>>for his SHREDDER, why then could he have the idea to let FRITZ [!!] play with
>>it??? No, Mike, if you can't judge such nonsense out of your own you have no
>>idea what testing conditions are all about! Excuse me, no insult intended. That
>>is simply incredible and Sandro already told you!
>
>But this is explained very well on M42's page. It's a computerchess specific
>book, not for a specific engine, but based on computerchess games and practical
>experience with several different engines. Maybe this book would even have been
>a better (and less controversial) replacement for CM9000.OBK in the King tests.
>
>>(...) Tell me what is going
>>on there??? Why must you defend such nonsense experiments???
>
>It is the nature of an experiment, that something unusual is being done (but
>carefully).


Exactly that wasn't the case. And admittedly by its author. Forget it as soon as
you can - it spoils your whole perception.





>I don't particularly defend that experiment - I wasn't aware that it
>needs defense :-))


What is if you will understand that the book is of major importance? Just a
question to begin with. Just as an assumption. What will you do then with all
these results? Or will you then come back to the only existing haven, called
science?



>
>Maybe this is all very normal from my viewpoint, because my own experiments were
>even much more unusual, sometimes (not primarily aiming for rankings, but more
>for other explorations like what is the value of the bishop pair in
>computerchess, how can engines make use of a large developement advantage,
>etc.).


But look, this is something totally different. Or did you intend with such
experiments to prove that chess is solved two days after XMas? Because THEN I
had something to criticise. This is the key! Michael42 is completely unaware of
what he's doing. He gets some results and is surprised. He didn't even suspect
such results, to be fair to him. And then he comes and says -- nothing <g>, but
Mike S. comes and says that to him this was clear already since for a whole
century that books are unimportant, _relatively_ unimportant. But Bob say that
books can have a crucial importance in tournaments! Duh!


>
>I've even played experimental engine games with odds. Fritz 7 can give odds of a
>rook to Doctor? 3.0 and still win:


Just ay naive question, no pin intended, is Doctor aware of the lacking Rook???
:)




>
>[Event "Turmvorgabe"]
>[Site "P3/700, 10'/40"]
>[Date "2003.01.01"]
>[Round "2"]
>[White "Fritz 7"]
>[Black "Doctor? 3.0"]
>[Result "1-0"]
>[SetUp "1"]
>[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/1NBQKBNR b Kkq - 0 1"]
>[PlyCount "98"]
>[EventDate "2003.01.01"]
>
>{W=11.4 ply; 370kN/s  B=15.9 ply; 57kN/s} 1... Nf6 {
>[%eval -510,19] [%emt 0:00:17]} 2. d4 {[%eval -487,12] [%emt 0:00:21]} e6 {
>[%eval -508,19] [%emt 0:00:22]} 3. Nf3 {[%eval -484,11] [%emt 0:00:18]} c5 {
>(d5) [%eval -509,17] [%emt 0:00:15]} 4. c4 {(e3) [%eval -478,10] [%emt 0:00:18]
>} Qa5+ {(cxd4) [%eval -542,17] [%emt 0:00:21]} 5. Nc3 {
>[%eval -462,11] [%emt 0:00:10]} Ne4 {(d5) [%eval -538,17] [%emt 0:00:22]} 6.
>Bd2 {(Dc2) [%eval -472,12] [%emt 0:00:19]} Nxc3 {[%eval -570,18] [%emt 0:00:15]
>} 7. Bxc3 {[%eval -475,12] [%emt 0:00:08]} Qxa2 {[%eval -572,18] [%emt 0:00:14]
>} 8. e3 {(e4) [%eval -469,12] [%emt 0:00:24]} Na6 {
>[%eval -573,17] [%emt 0:00:16]} 9. Bd3 {(d5) [%eval -481,11] [%emt 0:00:17]}
>cxd4 {[%eval -584,17] [%emt 0:00:16]} 10. exd4 {[%eval -478,11] [%emt 0:00:13]}
>Nb4 {[%eval -584,17] [%emt 0:00:13]} 11. Be2 {[%eval -500,12] [%emt 0:00:23]}
>d5 {(b5) [%eval -589,17] [%emt 0:00:13]} 12. c5 {[%eval -472,12] [%emt 0:00:27]
>} Nc6 {[%eval -589,18] [%emt 0:00:12]} 13. O-O {[%eval -475,11] [%emt 0:00:17]}
>g6 {(b6) [%eval -588,18] [%emt 0:00:14]} 14. b4 {
>(Lb5) [%eval -419,11] [%emt 0:00:11]} Bh6 {[%eval -559,16] [%emt 0:00:16]} 15.
>Qd3 {[%eval -394,11] [%emt 0:00:17]} Ne7 {(e5) [%eval -532,16] [%emt 0:00:14]}
>16. Ne5 {(Ta1) [%eval -256,11] [%emt 0:00:05]} f6 {
>[%eval -410,17] [%emt 0:00:14]} 17. Ra1 {[%eval -272,11] [%emt 0:00:06]} Qxa1+
>{[%eval -415,19] [%emt 0:00:18]} 18. Bxa1 {[%eval -291,12] [%emt 0:00:04]} fxe5
>{[%eval -418,19] [%emt 0:00:21]} 19. dxe5 {[%eval -297,12] [%emt 0:00:13]} a5 {
>(0-0) [%eval -423,19] [%emt 0:00:19]} 20. b5 {
>(bxa5) [%eval -303,13] [%emt 0:00:32]} Bg7 {
>(0-0) [%eval -431,17] [%emt 0:00:14]} 21. Qa3 {
>(Lg4) [%eval -306,13] [%emt 0:00:24]} Bd7 {(a4) [%eval -426,19] [%emt 0:00:19]}
>22. c6 {(Lb2) [%eval -294,12] [%emt 0:00:14]} bxc6 {
>[%eval -506,18] [%emt 0:00:12]} 23. b6 {[%eval -287,13] [%emt 0:00:12]} Bc8 {
>(Tb8) [%eval -489,17] [%emt 0:00:12]} 24. Qd6 {[%eval -259,12] [%emt 0:00:08]}
>Kf7 {(a4) [%eval -365,17] [%emt 0:00:12]} 25. Bb2 {
>(Dc7) [%eval -244,13] [%emt 0:00:18]} a4 {(Te8) [%eval -403,16] [%emt 0:00:14]}
>26. Ba3 {(Lg4) [%eval -234,12] [%emt 0:00:07]} Re8 {
>[%eval -328,17] [%emt 0:00:10]} 27. Qc7 {[%eval -241,11] [%emt 0:00:06]} Bf8 {
>[%eval -305,16] [%emt 0:00:12]} 28. Bd3 {(Lf1) [%eval -228,12] [%emt 0:00:21]}
>Kg8 {[%eval -306,16] [%emt 0:00:18]} 29. g4 {
>(b7) [%eval -222,11] [%emt 0:00:10]} Kg7 {(Kf7) [%eval -313,16] [%emt 0:00:12]}
>30. Kf1 {(Lf1) [%eval -206,11] [%emt 0:00:10]} Kf7 {
>(c5) [%eval -316,17] [%emt 0:00:10]} 31. Ke1 {
>(Kg1) [%eval -181,11] [%emt 0:00:15]} Bh6 {[%eval -325,17] [%emt 0:00:14]} 32.
>h4 {(Kf1) [%eval -166,11] [%emt 0:00:09]} Bg7 {
>(Lf8) [%eval -340,16] [%emt 0:00:09]} 33. Kd1 {
>(Lf1) [%eval -156,11] [%emt 0:00:10]} Bh6 {(Lf8) [%eval -351,16] [%emt 0:00:11]
>} 34. Ke1 {[%eval 0,13] [%emt 0:00:04]} Bf4 {[%eval -332,17] [%emt 0:00:09]}
>35. Kd1 {(Kf1) [%eval -125,12] [%emt 0:00:13]} Bh2 {
>[%eval -332,17] [%emt 0:00:13]} 36. Ke1 {[%eval -112,11] [%emt 0:00:08]} d4 {
>[%eval -261,16] [%emt 0:00:11]} 37. Kd1 {(Le4) [%eval -81,11] [%emt 0:00:11]}
>g5 {(Lf4) [%eval -216,15] [%emt 0:00:10]} 38. Bxh7 {
>(hxg5) [%eval 78,11] [%emt 0:00:07]} Ba6 {[%eval -33,15] [%emt 0:00:13]} 39. h5
>{[%eval 191,12] [%emt 0:00:13]} c5 {[%eval 11,14] [%emt 0:00:09]} 40. Bg6+ {
>[%eval 222,11] [%emt 0:00:06]} Kf8 {(Kg7) [%eval 172,16] [%emt 0:00:08]} 41.
>Bxc5 {(Lxe8) [%eval 350,11] [%emt 0:00:06]} d3 {
>(a3) [%eval 425,15] [%emt 0:00:17]} 42. Qd7 {
>(Dc6) [%eval 1069,11] [%emt 0:00:07]} Bc4 {[%eval 1225,18] [%emt 0:00:19]} 43.
>b7 {[%eval 1372,11] [%emt 0:00:08]} Bb3+ {[%eval 32756,17] [%emt 0:00:15]} 44.
>Kd2 {[%eval 1422,11] [%emt 0:00:05]} Bf4+ {[%eval 32757,9] [%emt 0:00:02]} 45.
>Kxd3 {[%eval 2350,11] [%emt 0:00:10]} Ba2 {
>(Tab8) [%eval 32758,9] [%emt 0:00:02]} 46. Bxe7+ {
>[%eval 32762,6] [%emt 0:00:01]} Kg8 {[%eval 32761,0] [%emt 0:00:00]} 47. Bh7+ {
>[%eval 32763,5] [%emt 0:00:02]} Kxh7 {[%eval 32764,1] [%emt 0:00:00]} 48. Bf8+
>{[%eval 32764,3] [%emt 0:00:01]} Re7 {[%eval 32765,0] [%emt 0:00:01]} 49. Qxe7+
>{[%eval 32765,2] [%emt 0:00:00]} Kg8 {(Kh8) [%eval 32766,0] [%emt 0:00:00]} 50.
>Qg7# {[%eval 32766,2] [%emt 0:00:01]} 1-0
>
>But from a total of 8 such games against Fritz 7 and Tiger 14, Doctor 3.0 won 5
>and lost 3, which means the rook more than compensates for the strength
>difference, in average.
>
>(Piece values have been expressed in Elo, and I wanted to see how the method to
>evaluate these by match results, works. But I didn't continue this.)
>
>>I would be interested if SHREDDER has an extra implementation in its engine?
>>That is what I would ask myself from a science view. But nobody seems to be
>>interested in such questions. The results show into that direction. But then I'm
>>a layman in CC [computerchess].
>
>In the developement advantage experiment mentioned, Shredder (5.32 at that time)
>did very well, too. For example a win with Black (!) against Tiger 14 from this
>position:
>
>[D]rnbqkbnr/pppppppp/8/8/3PP3/2NB1N2/PPPBQPPP/R3K2R b KQkq - 0 7
>
>(But Tiger 14 also won his game with Black against Shredder 5.32, in return!)
>
>Most probably, general priniciples of the early opening have been "implanted"
>very good, in Shredder.


Now what does this tell us? That I am a very talented computerchess lay? Don't
miss my little message about the alleged limes of the good of more depth...!
(Schroder/Theron et. al.)

Rolf


>OTOH, in the FRC rating list where games are played
>without books too, because these start from Fischerrandom positions, Shredder
>7.04 is sligthly behind Hiarcs 9 (!) and Fritz 8. This could indicate that it
>uses patterns which fit to the classic starting position better. But that are
>just assumptions; the 3 engines mentioned are within a small performance
>bandwidth of 0.6% only, on that rating list.
>http://www.beepworld.de/members53/frc-list/
>
>mfg.
>M.Scheidl



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.