Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test position and Crafty slow down

Author: Robert Hyatt

Date: 06:09:20 11/10/99

Go up one level in this thread


On November 10, 1999 at 04:45:53, Bernhard Bauer wrote:

>On November 10, 1999 at 04:07:49, Dann Corbit wrote:
>
>>On November 10, 1999 at 03:37:47, Bernhard Bauer wrote:
>>>On November 09, 1999 at 17:17:06, James Robertson wrote:
>>[snip]
>>>I gave this position to crafty and did a search Nd5. Here are the results for
>>>crafty16.19 and crafty17.0.
>>>
>>>Crafty16.19
>>>===========
>>>
>>>  12->  11.30  -0.07   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                       4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 Nd7 7.
>>>                       Ng5 Qc2 8. Ne6+ Ke7 9. Rd3 Qxa2 10.
>>>                       Nxg7
>>>  13    27.83   0.13   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                       4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                       Qf5 Ba6 8. Rc1 Qxc1 9. Rxc1 Rxc1+
>>>  13->  27.83   0.13   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                       4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                       Qf5 Ba6 8. Rc1 Qxc1 9. Rxc1 Rxc1+
>>>  14    44.83     ++   1. Nd5!!
>>>  14->   1:18   0.52   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                       4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                       Qf5 Ba6 8. Rc1 Qxc1 9. Rxc1 Rxc1+
>>>  15     2:12     ++   1. Nd5!!
>>>  15->   8:59   0.91   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                       4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                       Qf5 Ba6 8. Rc1 Qxc1 9. Rxc1 Rxc1+
>>> time=15:00  cpu=199%  mat=0  n=476806687  fh=96%  nps=529508
>>> ext-> checks=46001900 recaps=737487 pawns=154520 1rep=5107499 thrt:139967
>>> predicted=0  nodes=476806687  evals=31020644
>>> endgame tablebase-> probes done=0  successful=0
>>> SMP->  split=1274  stop=164  data=10/64  cpu=29:59  elap=15:00
>>>
>>>and
>>>crafty17.0
>>>==========
>>>
>>> 12->  15.23  -0.67   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 Nd7 7.
>>>                      Ng5 Qc2 8. Ne6+ Ke7 9. Rd3 Qxa2 10.
>>>                      Nxg7
>>> 13    30.92  -0.60   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                      Bg6 Qc5 8. e4 Nd7
>>> 13->  30.92  -0.60   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                      Bg6 Qc5 8. e4 Nd7
>>> 14    46.22     ++   1. Nd5!!
>>> 14->   1:35  -0.21   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Rc8 6. Nxf7 a5 7.
>>>                      Bg6 Qc5 8. e4 Nd7
>>> 15     3:43   0.00   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Ke7 6. Nxf7 Rc8 7.
>>>                      Qf5 Kf8 8. Qf4 Ke7 9. Qf5
>>> 15->   3:43   0.00   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Ke7 6. Nxf7 Rc8 7.
>>>                      Qf5 Kf8 8. Qf4 Ke7 9. Qf5
>>> 16    10:51   0.00   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Bxg5 6. Qh8+ Ke7
>>>                      7. Qxg7 Rf8 8. Qxg5+ Ke8 9. Rc1 Qd8
>>>                      10. Qf5 Nd7 11. a3 Qf6
>>> 16->  10:51   0.00   1. Nd5 exd5 2. Bxf6 Bxf6 3. Qxh7+ Kf8
>>>                      4. cxd5 d6 5. Bh5 Bxg5 6. Qh8+ Ke7
>>>                      7. Qxg7 Rf8 8. Qxg5+ Ke8 9. Rc1 Qd8
>>>                      10. Qf5 Nd7 11. a3 Qf6
>>>time=15:00  cpu=200%  mat=0  n=314623673  fh=93%  nps=349445
>>>ext-> checks=16736788 recaps=643735 pawns=162104 1rep=1411877 thrt:97472
>>>predicted=0  nodes=314623673  evals=206046602
>>>endgame tablebase-> probes done=0  successful=0
>>>hashing-> trans/ref=23%  pawn=2%  used=99%
>>>SMP->  split=1349  stop=171  data=10/64  cpu=30:01  elap=15:00
>>>
>>>Comparing these results show:
>>>
>>>              Crafty16.19   Crafty17.0
>>>score(15)         0.91          0.0
>>>nps             529508       349445
>>>
>>>So for this position crafty16.19 looks much better than crafty17.0.
>>>Crafty17.0 is 34% slower than crafty16.19, not to mention the evaluation.
>>>BTW computation was done on a 2xPIII 450MHz computer running WinNT4.0.
>>Actually, crafty 17.0 beat the pants off of 16.19.  It finished ply 15 at 3:43
>>compared to 8:59 so it looks to be much more than twice as fast.  The nodes mean
>>nothing compared to finishing a ply.
>
>IMHO finishing a ply means nothing compared to finding the right continuation.
>Please note that crafty16.19 gives a different line which looks mutch better.
>So I wouldn't say "crafty 17.0 beat the pants off of 16.19". Such a statement
>looks somewhat superficial.
>Anyway, up to now there is no known reason why the new crafty is significantly
>slower than 16.19. See the current discussion at the crafty mailing list.
>Kind regards
>Bernhard


As I mentioned on the mailing list, comparing NPS between two versions is a
good idea, but _not_ with mt=2 enabled.  There are too many variables, and the
nps will vary significantly.  mt=2 is the right way to _run_ tests, but it is
the wrong way to run if you want to compare nps, or time to finish a ply.  You
have to run the same test dozens of times and take the average to get reasonable
results...



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.