Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Chess program similarity experiment (Results)

Author: Robert Hyatt

Date: 19:44:01 01/29/99

Go up one level in this thread


On January 29, 1999 at 11:19:52, G.Mueller wrote:

>On January 29, 1999 at 10:45:51, Albrecht Heeffer wrote:
>
>>On January 29, 1999 at 09:23:15, G.Mueller wrote:
>>
>>>On January 29, 1999 at 08:28:26, Robert Hyatt wrote:
>>>
>>>>On January 29, 1999 at 02:31:21, Albrecht Heeffer wrote:
>>>>
>>>>>On January 28, 1999 at 15:56:40, Robert Hyatt wrote:
>>>>>
>>>>>>note that some of the positions end up in tablebases.  I don't know what they
>>>>>>used, but I used none, because of the format change from 15.20 to 16.0.  As a
>>>>>>result, I ran with no tablebase hits possible...
>>>>>
>>>>>We did use the old tablebases (72). In the log files you can see the probes.
>>>>>In fact we probably saved the Kallisto game from a draw by the tablesbases.
>>>>>The endgame was was winning but Bionic did not play the best moves until
>>>>>suddenly:
>>>>>
>>>>>               13     6.31  fhigh   Kd6!!
>>>>>               13->   7.20   6.62   Kd6
>>>>>               14     7.58  fhigh   Kd6!!
>>>>>               14->  13.08   7.01   Kd6
>>>>>               15    13.26  fhigh   Kd6!!
>>>>>               15    17.08  Mat28   Kd6 Kg7 Ke6 f5 Bf3 Kh6 Kf6 Kh7 Bc6
>>>>>                                    Kh6 Bd7 <HT>
>>>>>              time=23.65  cpu=196%  mat=2  n=9446180  fh=98%  nps=399415
>>>>>              ext-> checks=236796 recaps=13251 pawns=234693 1rep=184173
>>>>>              predicted=37  nodes=9446180  evals=3370257
>>>>>              endgame tablebase-> probes done=137165  successful=137159
>>>>>              hashing-> trans/ref=111%  pawn=99%  used=w29% b37%
>>>>>              SMP->  split=2503  stop=80  data=8/64  cpu=46.52  elap=23.65
>>>>>
>>>>>mate in 28 moves.
>>>>>
>>>>>>If someone has a PII/400 they can run on, I can run crafty on one processor at
>>>>>>1 minute per position, and they could run bionic the same way.  A PII/400 and
>>>>>>my xeon/400 are close to the same speed.  that would be the best comparison
>>>>>>unless one person wants to run both bionic _and_ wcrafty_15.20.exe (from my
>>>>>>ftp site) on the same box, which would be even better...
>>>>>>
>>>>>>I'd much prefer that to eliminate all variables.  the SMP code produces some
>>>>>>odd results at times, and my linux version is 15% slower than the corresponding
>>>>>>windows executable due to MSVC being a better compiler.  All in all, too many
>>>>>>different things...
>>>>>
>>>>>The results so far do not seem to indicate that Crafty 15.20 plays all
>>>>>the same moves as the Bionic games in the Dutch Open. I'm still wondering
>>>>>how you were able to reproduce all the right moves in three games with
>>>>>Crafty 16.1 on your hardware. Did you use SMP then?
>>>>>
>>>>>Albrecht Heeffer
>>>>
>>>>
>>>>No.  What I did was search for 10 minutes per position, and if the
>>>>programs matched after a minute or more, I counted it as a match.  I
>>>>then looked at the very few where they didn't match.  In a couple of
>>>>cases there were simple transpositions that evaluated to the same score
>>>>and pre-processing could affect that by changing the root move order.
>>>>If crafty's move and bionic's move had the same score, and roughly the
>>>>same PV I counted those as matches.  Finally, in a couple of cases it
>>>>was obvious that they had the same idea, just different ways to reach
>>>>that...  The pv's were similar but in different order (notably in a
>>>>couple of endgame positions where there were no real tactics to consider.)
>>>>
>>>>But notice that 'bionic' is not matching 'bionic-tournament' very well,
>>>>when you think about it.  The web site version matched 77% of the moves,
>>>>while version 15.20 matched 74%.  Everyone else is well back from that.
>>>>And I was searching faster than the bionic in the tournament, and the
>>>>one tested here, which is why I suggested someone run 15.20 and bionic
>>>>on the _same_ hardware and do this test.  That would be better...
>>>Hello Bob!
>>>
>>>You are right I do remark same on my single Xeon overclockt to 504 Mhz with 1MB
>>>2.level Cache, the downloadable bionic is never the same version that played in
>>>Dutch Championchip, it matches about 80%, Crafty 16.1 in about 75%, do not test
>>>with TB. A download from the originally Dutchversion would be very nice to
>>>answer all this difficult questions.
>>>In Dualuse (i test on a Quadxeon with mt=2) Bionic seems evident fast as Crafty
>>>16.1 on single it is slower. But a Quadmachine is not really good for testing a
>>>dual machine I know, but bionic is compiled with two CPUS.
>>>
>>>Best wishes to you Bob!
>>>G.Mueller
>>
>>After the tournament I burned a CD-R with the complete directory image
>>of all sources, executables and log files. The version on the website
>>'bionic41.exe' is ftp'd directly from the CD onto the web site. It _is_
>>the same version as used during both weekends of the tournament, I
>>can assure you. The SMP code must be very indeterministic if we can't
>>reproduce the moves somehow.
>>
>>Albrecht Heeffer
>
>Hello Albrecht!
>
>This is a possibilty maybe Bob can tell us, if it is so with his SMP technic.
>But if it is so the high match rate from Crafty 16_1 is very remarkable in my
>opinion. Maybe you can compile a single processor version and then we can
>compare to single versions from crafty and bionic, this would give more
>reproducable results I think.
>
>Best wishes
>G.Mueller



the results I gave bruce were crafty version 15.20... not 16.anything...
although I could run that test...

Your idea about the two programs on a single cpu is good, and one I have already
suggested.  A multiple-cpu compile (bionic or crafty) will run with one cpu with
no problems...  just don't do the mt=2 command.  It will then use just one cpu
and not act so funny...



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.