Author: Uri Blass
Date: 13:26:01 11/22/03
Go up one level in this thread
On November 22, 2003 at 15:56:08, Robert Hyatt wrote: >On November 22, 2003 at 01:01:35, Peter Kappler wrote: > >>On November 21, 2003 at 22:03:53, Robert Hyatt wrote: >> >>>On November 21, 2003 at 19:11:41, Will Singleton wrote: >>> >>>>On November 21, 2003 at 18:24:42, Robert Hyatt wrote: >>>> >>>>>On November 21, 2003 at 18:05:55, Joachim Rang wrote: >>>>> >>>>>>On November 21, 2003 at 16:47:52, Peter Berger wrote: >>>>>> >>>>>>>Following recent heated discussions I'd love to do a little testmatch. There are >>>>>>>different versions of the challenge online - I chose one that I can kind of >>>>>>>simulate myself with slower hardware. >>>>>>> >>>>>>>Junior 8.0.0.2 will play on a P233MMX, 32 MB RAM. >>>>>>>Crafty 19.4 will play on a PIV2.0GHz notebook, 1GHz RAM. >>>>>>> >>>>>>>Time control will be game in 2 hours with 10 seconds increment/move. The match >>>>>>>will be done like older FIDE world championship matches - the first one to win 6 >>>>>>>games wins the match, draws won't count. >>>>>>> >>>>>>>Junior uses 16MB Hash, 3+4 men tablebases, 1MB cache, junior8.ctg. >>>>>>>Crafty uses 384MB Hash, 64MB hashp, 3+4 men tablebases, 32MB cache, own book, >>>>>>>aware of playing a computer. >>>>>>> >>>>>>>Compairing setups with Crafty bench (hash 12M, hashp 3M, cache 1M on the slower >>>>>>>one) suggests a speed difference factor of about 10.5 in raw nodes per seconds >>>>>>>and 11.0 in "SMP time to-ply-measurement" between the two computers. >>>>>>>As the Junior Mark doesn't work on the slower one and Junior chooses to search >>>>>>>different depths on both in the starting position, I can't really give a number. >>>>>>>The difference seems to be slightly lower for Junior though, sth like 9.0 maybe. >>>>>>> >>>>>>>Saying that the faster computer is about 10 times faster shouldn't be too wrong. >>>>>>> >>>>>>>That's also clearly an upper-bound for faster hardware Crafty could reasonably >>>>>>>come up to compete with against a single-CPU opponent in a current competition >>>>>>>on fast computers IMHO - the speedup demands 16 CPUs I guess, and I don't know >>>>>>>if Crafty can really scale that well. >>>>>>> >>>>>>>With this setup Crafty should be the clear favourite I suppose. >>>>>>> >>>>>>>Crafty won the toss and will have the white pieces in the first game. >>>>>>> >>>>>>>Peter >>>>>> >>>>>> >>>>>>nice test indeed. I think crafty will win. >>>>>> >>>>>>regards Joachim >>>>> >>>>>If it loses I should obviously retire from computer chess due to gross >>>>>incompetence. >>>> >>>>You already have two people (above) who think Junior has a chance to win, so I >>>>suppose you should at least start checking the social security balance. Me, I >>>>think Crafty will win easily, perhaps 6-0 (not counting draws). >>>> >>>>Will >>> >>>I'd expect 5-1 or 6-0 at 10:1 time odds, discounting quite a few draws >>>that are expected... >>> >>>As for who thinks that a 10:1 time odds is not enough, I suspect they have >>>not "gotten dirty" by writing something that is serious. 10:1 is quite a >>>bonus. 100:1 is even better... >>> >>>If I couldn't win at 10:1 it would be time to hang it up, IMHO, no matter >>>how much or little time I get to spend on it... >>> >> >> >>When did the time odds change from 100x to 10x? > >There are _two_ things going on, which is causing confusion. Amir and I >are talking about 100:1 time odds. The match being played on CCC right >now is using a 10:1 handicap by using a slow pentium 233 vs a new machine >that is about 10x faster. > >This is just a "proof of concept" to see if 100:1 will be significant or >not. > > >> >>The SSDF rating list shows Junior ahead by 25 ELO even when Crafty runs on a >>machine that is 4x faster. >> >>Junior 7.0 128MB K6-2 450 MHz 2639 >>Crafty 18.12/CB 256MB Athlon 1200 MHz 2615 >> >>(My Athlon 1400 is 4.5x faster than my K6-3/450, so I think 4x is about right >>for the two machines above.) >> >>The same rating list shows that Junior 8.0 is 100 pts stronger than Junior 7. I >>don't know how much progress Crafty has made since version 18.12, but I would >>guess that it's less than 100 pts. >> >>You'll also likely be facing Junior's tournament opening book - potentially a >>huge advantage for Junior. > >No, the book is _specifically_ eliminated. This is testing speed vs speed, >not book vs book which is uninteresting in time odds. If you saw my suggestion >for rules, "if either program is -1 or worse out of book, the game is aborted >and a new one started with the same colors, to eliminate book traps that should >not matter when comparing two engines at time odds." book traps are not eliminated in this way because it is possible to get position that both programs do not understand but can find the right moves because it is clear that there is no good alternative and only some moves after they are out of book one program understand it is -1 worse. The only way to eliminate book is to play something similiar to the nunn match or in other words after a game is finished to play the same opening with opposite colors. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.