Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Chess program similarity experiment (Results)

Author: Albrecht Heeffer

Date: 07:45:51 01/29/99

Go up one level in this thread


On January 29, 1999 at 09:23:15, G.Mueller wrote:

>On January 29, 1999 at 08:28:26, Robert Hyatt wrote:
>
>>On January 29, 1999 at 02:31:21, Albrecht Heeffer wrote:
>>
>>>On January 28, 1999 at 15:56:40, Robert Hyatt wrote:
>>>
>>>>note that some of the positions end up in tablebases.  I don't know what they
>>>>used, but I used none, because of the format change from 15.20 to 16.0.  As a
>>>>result, I ran with no tablebase hits possible...
>>>
>>>We did use the old tablebases (72). In the log files you can see the probes.
>>>In fact we probably saved the Kallisto game from a draw by the tablesbases.
>>>The endgame was was winning but Bionic did not play the best moves until
>>>suddenly:
>>>
>>>               13     6.31  fhigh   Kd6!!
>>>               13->   7.20   6.62   Kd6
>>>               14     7.58  fhigh   Kd6!!
>>>               14->  13.08   7.01   Kd6
>>>               15    13.26  fhigh   Kd6!!
>>>               15    17.08  Mat28   Kd6 Kg7 Ke6 f5 Bf3 Kh6 Kf6 Kh7 Bc6
>>>                                    Kh6 Bd7 <HT>
>>>              time=23.65  cpu=196%  mat=2  n=9446180  fh=98%  nps=399415
>>>              ext-> checks=236796 recaps=13251 pawns=234693 1rep=184173
>>>              predicted=37  nodes=9446180  evals=3370257
>>>              endgame tablebase-> probes done=137165  successful=137159
>>>              hashing-> trans/ref=111%  pawn=99%  used=w29% b37%
>>>              SMP->  split=2503  stop=80  data=8/64  cpu=46.52  elap=23.65
>>>
>>>mate in 28 moves.
>>>
>>>>If someone has a PII/400 they can run on, I can run crafty on one processor at
>>>>1 minute per position, and they could run bionic the same way.  A PII/400 and
>>>>my xeon/400 are close to the same speed.  that would be the best comparison
>>>>unless one person wants to run both bionic _and_ wcrafty_15.20.exe (from my
>>>>ftp site) on the same box, which would be even better...
>>>>
>>>>I'd much prefer that to eliminate all variables.  the SMP code produces some
>>>>odd results at times, and my linux version is 15% slower than the corresponding
>>>>windows executable due to MSVC being a better compiler.  All in all, too many
>>>>different things...
>>>
>>>The results so far do not seem to indicate that Crafty 15.20 plays all
>>>the same moves as the Bionic games in the Dutch Open. I'm still wondering
>>>how you were able to reproduce all the right moves in three games with
>>>Crafty 16.1 on your hardware. Did you use SMP then?
>>>
>>>Albrecht Heeffer
>>
>>
>>No.  What I did was search for 10 minutes per position, and if the
>>programs matched after a minute or more, I counted it as a match.  I
>>then looked at the very few where they didn't match.  In a couple of
>>cases there were simple transpositions that evaluated to the same score
>>and pre-processing could affect that by changing the root move order.
>>If crafty's move and bionic's move had the same score, and roughly the
>>same PV I counted those as matches.  Finally, in a couple of cases it
>>was obvious that they had the same idea, just different ways to reach
>>that...  The pv's were similar but in different order (notably in a
>>couple of endgame positions where there were no real tactics to consider.)
>>
>>But notice that 'bionic' is not matching 'bionic-tournament' very well,
>>when you think about it.  The web site version matched 77% of the moves,
>>while version 15.20 matched 74%.  Everyone else is well back from that.
>>And I was searching faster than the bionic in the tournament, and the
>>one tested here, which is why I suggested someone run 15.20 and bionic
>>on the _same_ hardware and do this test.  That would be better...
>Hello Bob!
>
>You are right I do remark same on my single Xeon overclockt to 504 Mhz with 1MB
>2.level Cache, the downloadable bionic is never the same version that played in
>Dutch Championchip, it matches about 80%, Crafty 16.1 in about 75%, do not test
>with TB. A download from the originally Dutchversion would be very nice to
>answer all this difficult questions.
>In Dualuse (i test on a Quadxeon with mt=2) Bionic seems evident fast as Crafty
>16.1 on single it is slower. But a Quadmachine is not really good for testing a
>dual machine I know, but bionic is compiled with two CPUS.
>
>Best wishes to you Bob!
>G.Mueller

After the tournament I burned a CD-R with the complete directory image
of all sources, executables and log files. The version on the website
'bionic41.exe' is ftp'd directly from the CD onto the web site. It _is_
the same version as used during both weekends of the tournament, I
can assure you. The SMP code must be very indeterministic if we can't
reproduce the moves somehow.

Albrecht Heeffer



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.