Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Chess program similarity experiment (Results)

Author: G.Mueller

Date: 08:19:52 01/29/99

Go up one level in this thread


On January 29, 1999 at 10:45:51, Albrecht Heeffer wrote:

>On January 29, 1999 at 09:23:15, G.Mueller wrote:
>
>>On January 29, 1999 at 08:28:26, Robert Hyatt wrote:
>>
>>>On January 29, 1999 at 02:31:21, Albrecht Heeffer wrote:
>>>
>>>>On January 28, 1999 at 15:56:40, Robert Hyatt wrote:
>>>>
>>>>>note that some of the positions end up in tablebases.  I don't know what they
>>>>>used, but I used none, because of the format change from 15.20 to 16.0.  As a
>>>>>result, I ran with no tablebase hits possible...
>>>>
>>>>We did use the old tablebases (72). In the log files you can see the probes.
>>>>In fact we probably saved the Kallisto game from a draw by the tablesbases.
>>>>The endgame was was winning but Bionic did not play the best moves until
>>>>suddenly:
>>>>
>>>>               13     6.31  fhigh   Kd6!!
>>>>               13->   7.20   6.62   Kd6
>>>>               14     7.58  fhigh   Kd6!!
>>>>               14->  13.08   7.01   Kd6
>>>>               15    13.26  fhigh   Kd6!!
>>>>               15    17.08  Mat28   Kd6 Kg7 Ke6 f5 Bf3 Kh6 Kf6 Kh7 Bc6
>>>>                                    Kh6 Bd7 <HT>
>>>>              time=23.65  cpu=196%  mat=2  n=9446180  fh=98%  nps=399415
>>>>              ext-> checks=236796 recaps=13251 pawns=234693 1rep=184173
>>>>              predicted=37  nodes=9446180  evals=3370257
>>>>              endgame tablebase-> probes done=137165  successful=137159
>>>>              hashing-> trans/ref=111%  pawn=99%  used=w29% b37%
>>>>              SMP->  split=2503  stop=80  data=8/64  cpu=46.52  elap=23.65
>>>>
>>>>mate in 28 moves.
>>>>
>>>>>If someone has a PII/400 they can run on, I can run crafty on one processor at
>>>>>1 minute per position, and they could run bionic the same way.  A PII/400 and
>>>>>my xeon/400 are close to the same speed.  that would be the best comparison
>>>>>unless one person wants to run both bionic _and_ wcrafty_15.20.exe (from my
>>>>>ftp site) on the same box, which would be even better...
>>>>>
>>>>>I'd much prefer that to eliminate all variables.  the SMP code produces some
>>>>>odd results at times, and my linux version is 15% slower than the corresponding
>>>>>windows executable due to MSVC being a better compiler.  All in all, too many
>>>>>different things...
>>>>
>>>>The results so far do not seem to indicate that Crafty 15.20 plays all
>>>>the same moves as the Bionic games in the Dutch Open. I'm still wondering
>>>>how you were able to reproduce all the right moves in three games with
>>>>Crafty 16.1 on your hardware. Did you use SMP then?
>>>>
>>>>Albrecht Heeffer
>>>
>>>
>>>No.  What I did was search for 10 minutes per position, and if the
>>>programs matched after a minute or more, I counted it as a match.  I
>>>then looked at the very few where they didn't match.  In a couple of
>>>cases there were simple transpositions that evaluated to the same score
>>>and pre-processing could affect that by changing the root move order.
>>>If crafty's move and bionic's move had the same score, and roughly the
>>>same PV I counted those as matches.  Finally, in a couple of cases it
>>>was obvious that they had the same idea, just different ways to reach
>>>that...  The pv's were similar but in different order (notably in a
>>>couple of endgame positions where there were no real tactics to consider.)
>>>
>>>But notice that 'bionic' is not matching 'bionic-tournament' very well,
>>>when you think about it.  The web site version matched 77% of the moves,
>>>while version 15.20 matched 74%.  Everyone else is well back from that.
>>>And I was searching faster than the bionic in the tournament, and the
>>>one tested here, which is why I suggested someone run 15.20 and bionic
>>>on the _same_ hardware and do this test.  That would be better...
>>Hello Bob!
>>
>>You are right I do remark same on my single Xeon overclockt to 504 Mhz with 1MB
>>2.level Cache, the downloadable bionic is never the same version that played in
>>Dutch Championchip, it matches about 80%, Crafty 16.1 in about 75%, do not test
>>with TB. A download from the originally Dutchversion would be very nice to
>>answer all this difficult questions.
>>In Dualuse (i test on a Quadxeon with mt=2) Bionic seems evident fast as Crafty
>>16.1 on single it is slower. But a Quadmachine is not really good for testing a
>>dual machine I know, but bionic is compiled with two CPUS.
>>
>>Best wishes to you Bob!
>>G.Mueller
>
>After the tournament I burned a CD-R with the complete directory image
>of all sources, executables and log files. The version on the website
>'bionic41.exe' is ftp'd directly from the CD onto the web site. It _is_
>the same version as used during both weekends of the tournament, I
>can assure you. The SMP code must be very indeterministic if we can't
>reproduce the moves somehow.
>
>Albrecht Heeffer

Hello Albrecht!

This is a possibilty maybe Bob can tell us, if it is so with his SMP technic.
But if it is so the high match rate from Crafty 16_1 is very remarkable in my
opinion. Maybe you can compile a single processor version and then we can
compare to single versions from crafty and bionic, this would give more
reproducable results I think.

Best wishes
G.Mueller



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.