Author: Robert Hyatt
Date: 07:24:12 08/29/98
Go up one level in this thread
On August 29, 1998 at 04:40:12, Moritz Berger wrote:
>On August 28, 1998 at 22:10:49, Robert Hyatt wrote:
>
>>I don't want to turn this into a "mine is bigger than yours"
>>argument, and didn't mean for my post to sound like that.
>>
>>However, some more data:
>>
>>you just went +2, =1, -3 against singacrafty... which was my
>>point...
>
>Hey, if you can sound apologetic, I can, too ;-)
>
>1. It should have been +3 =1 -2 from my point of view. Mouse-slip in a +3
>position to blame:
>
>[Event ICC 5 4 08/28/1998 ]
>[Site Internet Chess Club ]
>[Date 1998.08.28 ]
>[Round - ]
>[White dustbin ]
>[Black singacrafty ]
>[Result 0-1 ]
>[WhiteElo 2702 ]
>[BlackElo 2829 ]
>[ECO B92 ]
>[NIC SI.11 ]
>[LongECO Sicilian: Najdorf, Opo\v censky variation ]
> 1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 a6 5. Nc3 Nf6
> 6. Be2 e5 7. Nb3 Be7 8. O-O O-O 9. Be3 Be6 10. Qd2 Nbd7
>11. a4 Rc8 12. a5 Qc7 13. Rfd1 Rfd8 14. Qe1 Nc5 15. Nxc5 dxc5
>16. Rxd8+ Qxd8 17. Qd2 Qf8 18. Rd1 c4 19. Bb6 Nd7 20. Nd5 Bxd5
>21. Qxd5 Nxb6 22. axb6 Rc6 23. Qxe5 Rxb6 24. Bxc4 Bf6 25. Qd5 Rxb2
>26. e5 Be7 27. Bb3 b5 28. Qc6 Bb4 29. Qxa6 Qe8 30. e6 fxe6
>31. Bxe6+ Kf8 32. Qb7 Be7 33. Bb3 h6 34. Qd5 Rxb3 35. c3 Rxc3
>{White resigns} 0-1
>
>35 c2xb3, of course ...
>
>2. I played 5 4 games with Junior at 5 0 to avoid getting into time trouble.
>That's a +4" time bonus for the automated Crafty vs. manual operator (who also
>sometimes mismouses at past 2am ...). Unfortunately, Singacrafty only allows
>tc<7' or tc 15 0, both of which (as I am eager to point out) are a disadvantage
>for any manual program (opposed to e.g. 2 12 where I can play 2 10 and lose much
>less time relative to the automated opponent).
>
>3. Singacrafty runs on a dual PII-400, me on a single PII-400.
>
>4. I used FritzPower books without any learning information. Singacrafty does
>total inclusive learning when it plays unattended on the server, also against
>other Junior's (e.g. ban).
>
>2+3 add up to an effective 3-4 fold hardware speed advantage for Singacrafty ...
>Still, if I hadn't messed up that capture move in a won position, the result
>would have been in favour of Junior.
I don't follow the 4x hardware advantage. 1. the dual PII/400 is about 1.4x-
1.5x faster than a single processor. This has been discussed here many times
and is caused by search overhead that can't be eliminated, where about 30% of
each processor (in a parallel environment) is wasted effort. 2. The automated
interface issue is also not clear... because of the following: (1) I know of
operators that play 0 3 and 0 4 *bullet* using a manual program and they cope
pretty well; (2) I've played 5 0 manually for hundreds of games early in the
development phase of crafty and was able to play 80-100 move games with no
problem at all, not even using a mouse. and (3) manual lets you force moves
quicker when the move is obvious, or let it run a little longer when it is
more complicated (don't know if you do this or not, but if you do, this is a
far bigger advantage than the autointerface time savings), while the auto
program has to do *everything* by itself...
IE I'd buy a 1.5x hardware advantage, but I believe that everything else washes
out depending on how you operate..
>If you want to use ICC as an argument, please use the established rating for ban
>(on PII-333 AFAIR) as a fair measure of J5s playing strength. Hint: You rarely
>see any manual program with a 3000+ rating ...
>
junior (ban) isn't "manual" on icc... Amir has an automated interface, and has
for at least a couple of years, so it plays just like the rest of us there...
>
>> The Nunn test, I can't begin to explain until I have
>>a chance to look at the games. But 16 losses and 4 draws means
>>at least a 400 point rating difference, and that I don't buy under
>>any conceivable circumstances.
>>
>>I didn't notice the version of
>>Crafty you used,
>
>15.17 Fritz engine.
>
>> but most any version generally has some interesting
>>bugs, because it is always in a state of strong flux, except for the
>>periods of time leading up to a major event like the WMCCC where I
>>stop adding, and only fix what is already broken. So, in general, I
>>would always expect a current version to do worse than I would think
>>statistics would suggest, because we are comparing a polished engine
>>to one that just had (within a week or two) major changes...
>
>So why does +16 =4 -0 sound implausible to you?
>
because I *never* see such results on a server, excepting for the period of
July-early august where there was a really severe bug that was producing
random evaluation scores... But that version was never released...
>>But even then, +400 is way out of the normal I would expect... As the
>>results against SingaCrafty shows... I've never doubted Junior's strength,
>>nor that of Fritz. But there must be "reason" in such reporting, to avoid
>>the wrong impression.
>
>Many reasons. To begin with, Dirk had 80% scores vs. Genius 5 at Blitz. I had
>80% scores vs. Rebel 9 at Blitz, same with M-Chess 7.1. Now 90% vs. Crafty
>doesn't sound that bad any longer ...
doesn't sound bad in that light, no. But the overall results sound fishy in
any circumstance... I'll know more once the final version hits ICC regularly,
as I try to avoid drawing conclusions from "ban" there since there is no way
to tell what is "good" and what has "bugs". (Yes, he has bugs too... based on
successive strings of losses)..
>
>> IE I personally watched Junior play some positionally
>>poor moves that none of the other programs being used to analyzed would even
>>consider.
>
>Oh well, the same applies at least at much for crafty. Foggy argument, if I may
>say so.
sure... didn't mean to imply otherwise, other than to say that IMHO, from the
games I've seen so far, Junior doesn't represent a new "high-water mark" in
chess. IE it did much worse against Yudasin at fast games than did Rebel vs
Anand... and Rebel's opponent was much stronger. It may just be that the
Junior "approach" works very well against computers, because of the speed, while
it doesn't fare so well against humans, where judgement is just as important as
raw speed...
IE it may well behave like Fritz... A computer killer that makes too many
mistakes (aggressive mistakes) against strong humans...
>
>> It seems very fast. And that is one good way to write a program
>>that thrashes other programs... although how such approaches work against
>>strong humans remains to be seen...
>>
>>One thing I did note is that Crafty's time utilization seems haywire in the
>>games... you didn't say how you played them, but if you are playing on the
>>same machine, that will cause problems from my end for sure, as time
>>allocation depends on "pondering"... if it is off, it will get into time
>>difficulties later in the game, every time...
>
>Fritz engines in internal engine match always run without PB (and yes, I clearly
>stated this). I deliberately used 25'+12" to avoid any time trouble situation
>caused by time management problems ... Crafty's time allocation looked similar
>enough to Junior's (or anybody elses) that I didn't suspect any problems ...
>
>I want to be fair and so I will replay the Nunn match on 2 machines, Crafty gets
>the PII-400 as I think that after the 8 40/120 Nunn games I played (+1 =0 -7
>from Crafty's point of view) it will have trouble to survive even against Junior
>on P233MMX (in this particular test, with 40/120 time controls).
>
>Sorry, you don't have to defend Crafty here. It does that itself on ICC, 20h a
>day, 360+ days a year, and it doesn't perform bad (I really liked the recent
>results vs. Diepx). As I explained, that's why I was even more surprised with
>Juniors outstanding performance against Crafty...
>
>Moritz
DiepX is becoming very strong... Vincent's been working hard. He rubs people
the wrong way from time to time, but he is getting results, for sure... And I
am not defending Crafty so much as simply trying to encourage everyone to beware
of only computer-vs-computer results. As I have said many times, a program that
does well against other computers is/can be a far different engine than that
which does well against computers.
And to draw conclusions from playing "Crafty" (either mine or one that is on
the ftp machine) is a real mistake. Some examples: 15.17 was released to fix
a bug in book move selection, at the request of "data" on ICC. I later found
that the "black" side is really broken in attack scoring... it will always
think that the black king is somewhere between a2 and h2, due to a gross bug.
Similar things happen in every version, although I hope that each version is
better than the previous one, except in cases like this. The "right" version
to draw real conclusions from is a wmccc-type version that has been only cleaned
up without adding new (possibly severe) bugs. A commercial engine comes out
once a year, after months of testing, beta testing, and tuning. Crafty is a far
different animal, with most of the 15.x versions being released only to get the
SMP search stuff up and working reliably (main.c comments are a good explanation
of this). But any of the versions are always suspect until the WMCCC tuning
cycle completes...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.