Author: Robert Hyatt
Date: 18:18:26 11/22/03
Go up one level in this thread
On November 22, 2003 at 16:26:01, Uri Blass wrote: >On November 22, 2003 at 15:56:08, Robert Hyatt wrote: > >>On November 22, 2003 at 01:01:35, Peter Kappler wrote: >> >>>On November 21, 2003 at 22:03:53, Robert Hyatt wrote: >>> >>>>On November 21, 2003 at 19:11:41, Will Singleton wrote: >>>> >>>>>On November 21, 2003 at 18:24:42, Robert Hyatt wrote: >>>>> >>>>>>On November 21, 2003 at 18:05:55, Joachim Rang wrote: >>>>>> >>>>>>>On November 21, 2003 at 16:47:52, Peter Berger wrote: >>>>>>> >>>>>>>>Following recent heated discussions I'd love to do a little testmatch. There are >>>>>>>>different versions of the challenge online - I chose one that I can kind of >>>>>>>>simulate myself with slower hardware. >>>>>>>> >>>>>>>>Junior 8.0.0.2 will play on a P233MMX, 32 MB RAM. >>>>>>>>Crafty 19.4 will play on a PIV2.0GHz notebook, 1GHz RAM. >>>>>>>> >>>>>>>>Time control will be game in 2 hours with 10 seconds increment/move. The match >>>>>>>>will be done like older FIDE world championship matches - the first one to win 6 >>>>>>>>games wins the match, draws won't count. >>>>>>>> >>>>>>>>Junior uses 16MB Hash, 3+4 men tablebases, 1MB cache, junior8.ctg. >>>>>>>>Crafty uses 384MB Hash, 64MB hashp, 3+4 men tablebases, 32MB cache, own book, >>>>>>>>aware of playing a computer. >>>>>>>> >>>>>>>>Compairing setups with Crafty bench (hash 12M, hashp 3M, cache 1M on the slower >>>>>>>>one) suggests a speed difference factor of about 10.5 in raw nodes per seconds >>>>>>>>and 11.0 in "SMP time to-ply-measurement" between the two computers. >>>>>>>>As the Junior Mark doesn't work on the slower one and Junior chooses to search >>>>>>>>different depths on both in the starting position, I can't really give a number. >>>>>>>>The difference seems to be slightly lower for Junior though, sth like 9.0 maybe. >>>>>>>> >>>>>>>>Saying that the faster computer is about 10 times faster shouldn't be too wrong. >>>>>>>> >>>>>>>>That's also clearly an upper-bound for faster hardware Crafty could reasonably >>>>>>>>come up to compete with against a single-CPU opponent in a current competition >>>>>>>>on fast computers IMHO - the speedup demands 16 CPUs I guess, and I don't know >>>>>>>>if Crafty can really scale that well. >>>>>>>> >>>>>>>>With this setup Crafty should be the clear favourite I suppose. >>>>>>>> >>>>>>>>Crafty won the toss and will have the white pieces in the first game. >>>>>>>> >>>>>>>>Peter >>>>>>> >>>>>>> >>>>>>>nice test indeed. I think crafty will win. >>>>>>> >>>>>>>regards Joachim >>>>>> >>>>>>If it loses I should obviously retire from computer chess due to gross >>>>>>incompetence. >>>>> >>>>>You already have two people (above) who think Junior has a chance to win, so I >>>>>suppose you should at least start checking the social security balance. Me, I >>>>>think Crafty will win easily, perhaps 6-0 (not counting draws). >>>>> >>>>>Will >>>> >>>>I'd expect 5-1 or 6-0 at 10:1 time odds, discounting quite a few draws >>>>that are expected... >>>> >>>>As for who thinks that a 10:1 time odds is not enough, I suspect they have >>>>not "gotten dirty" by writing something that is serious. 10:1 is quite a >>>>bonus. 100:1 is even better... >>>> >>>>If I couldn't win at 10:1 it would be time to hang it up, IMHO, no matter >>>>how much or little time I get to spend on it... >>>> >>> >>> >>>When did the time odds change from 100x to 10x? >> >>There are _two_ things going on, which is causing confusion. Amir and I >>are talking about 100:1 time odds. The match being played on CCC right >>now is using a 10:1 handicap by using a slow pentium 233 vs a new machine >>that is about 10x faster. >> >>This is just a "proof of concept" to see if 100:1 will be significant or >>not. >> >> >>> >>>The SSDF rating list shows Junior ahead by 25 ELO even when Crafty runs on a >>>machine that is 4x faster. >>> >>>Junior 7.0 128MB K6-2 450 MHz 2639 >>>Crafty 18.12/CB 256MB Athlon 1200 MHz 2615 >>> >>>(My Athlon 1400 is 4.5x faster than my K6-3/450, so I think 4x is about right >>>for the two machines above.) >>> >>>The same rating list shows that Junior 8.0 is 100 pts stronger than Junior 7. I >>>don't know how much progress Crafty has made since version 18.12, but I would >>>guess that it's less than 100 pts. >>> >>>You'll also likely be facing Junior's tournament opening book - potentially a >>>huge advantage for Junior. >> >>No, the book is _specifically_ eliminated. This is testing speed vs speed, >>not book vs book which is uninteresting in time odds. If you saw my suggestion >>for rules, "if either program is -1 or worse out of book, the game is aborted >>and a new one started with the same colors, to eliminate book traps that should >>not matter when comparing two engines at time odds." > >book traps are not eliminated in this way because it is possible to get position >that both programs do not understand but can find the right moves because it is >clear that there is no good alternative and only some moves after they are out >of book one program understand it is -1 worse. Not much I can do about this, except that if both think it is pretty even, then a deep trap is just something that happens. However, if it is a sac that neither program understands, then one will be +1 and the other will be -1, and that's enough to kick the opening out and start over. > >The only way to eliminate book is to play something similiar to the nunn match >or in other words after a game is finished to play the same opening with >opposite colors. Or, play an opening, then play the same opening with colors reversed... Of course one could have more book than the other, but that's ok... > >Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.