Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Test your program

Author: Robert Hyatt

Date: 08:06:28 05/05/01

Go up one level in this thread


On May 05, 2001 at 01:29:40, Uri Blass wrote:

>On May 05, 2001 at 00:53:39, Robert Hyatt wrote:
>
>>On May 05, 2001 at 00:20:16, Peter Kappler wrote:
>>
>>>On May 04, 2001 at 17:25:55, Robert Hyatt wrote:
>>>
>>>>On May 04, 2001 at 16:29:35, Uri Blass wrote:
>>>>
>>>>>On May 04, 2001 at 14:49:57, Robert Hyatt wrote:
>>>>>
>>>>>>On May 04, 2001 at 14:10:59, Uri Blass wrote:
>>>>>>
>>>>>>>On May 04, 2001 at 13:41:08, Robert Hyatt wrote:
>>>>>>>
>>>>>>>>On May 04, 2001 at 13:33:55, Uri Blass wrote:
>>>>>>>>
>>>>>>>>>On May 04, 2001 at 13:20:51, Dann Corbit wrote:
>>>>>>>>>
>>>>>>>>>>On May 04, 2001 at 10:52:52, Robert Hyatt wrote:
>>>>>>>>>>
>>>>>>>>>>>On May 03, 2001 at 21:03:58, Dann Corbit wrote:
>>>>>>>>>>>
>>>>>>>>>>>>On May 03, 2001 at 18:51:08, Eduard Nemeth wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>               12    54.72     --   1. ... Qxa3
>>>>>>>>>>>>               12     2:00  -3.04   1. ... Qxa3 2. Bf6 Rfd8 3. Bxd8 Rxd8
>>>>>>>>>>>>                                    4. Rd1 Rd5 5. Qe4 g6 6. Qb4 Qa6 7.
>>>>>>>>>>>>                                    Qf4
>>>>>>>>>>>>               12->   3:19  -3.04   1. ... Qxa3 2. Bf6 Rfd8 3. Bxd8 Rxd8
>>>>>>>>>>>>                                    4. Rd1 Rd5 5. Qe4 g6 6. Qb4 Qa6 7.
>>>>>>>>>>>>                                    Qf4
>>>>>>>>>>>>               13     3:43     --   1. ... Qxa3
>>>>>>>>>>>>               13     5:06   0.00   1. ... Qxa3 2. Bf6 Rfc8 3. Bxg7 Kxg7
>>>>>>>>>>>>                                    4. Qf6+ Kf8 5. Rf1 Rc7 6. Bg6 Re8 7.
>>>>>>>>>>>>                                    Qh8+ Ke7 8. Qf6+ Kf8
>>>>>>>>>>>>               13    11:27     ++   1. ... b4!!
>>>>>>>>>>>>               13    13:05  -0.76   1. ... b4 2. cxb4 Qd5 3. Rf1 Rae8 4.
>>>>>>>>>>>>                                    Bf2 f5 5. exf6 Qxg2 6. Bd4 Rf7 7. Bxa7
>>>>>>>>>>>>                                    Qd5
>>>>>>>>>>>>               13    13:40     ++   1. ... Rfe8!!
>>>>>>>>>>>>               13    14:40  -2.30   1. ... Rfe8 2. Rf1 Qf8 3. Qe4 g6 4.
>>>>>>>>>>>>                                    Kb2 Rec8 5. Bf6 Rc7 6. Qf4 Rd7 7. h4
>>>>>>>>>>>>                                    a5 8. Be4
>>>>>>>>>>>>               13    15:40  -2.63   1. ... Rfc8 2. Qe4 g6 3. Rf1 Qf8 4.
>>>>>>>>>>>>                                    Kb2 Rc7 5. Qg4 c5 6. Bxb5 Rb8 7. g3
>>>>>>>>>>>>                                    Be4
>>>>>>>>>>>>               13->  16:21  -2.63   1. ... Rfc8 2. Qe4 g6 3. Rf1 Qf8 4.
>>>>>>>>>>>>                                    Kb2 Rc7 5. Qg4 c5 6. Bxb5 Rb8 7. g3
>>>>>>>>>>>>                                    Be4
>>>>>>>>>>>>              time=16:39  cpu=100%  mat=-3  n=596994678  fh=91%  nps=597k
>>>>>>>>>>>>              ext-> chk=29940411 cap=1174947 pp=514823 1rep=4254557 mate=419833
>>>>>>>>>>>>              predicted=0  nodes=596994678  evals=102285376
>>>>>>>>>>>>              endgame tablebase-> probes done=0  successful=0
>>>>>>>>>>>>Black(1): quit
>>>>>>>>>>>>execution complete.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>Mine looks a bit different on the quad.  2:18 to drop Qxa3.  Note that I used
>>>>>>>>>>>hash=192M for the run...
>>>>>>>>>>>
>>>>>>>>>>>               12    38.66     --   1. ... Qxa3
>>>>>>>>>>>               12     1:27  -1.65   1. ... Qxa3 2. Bf6 Rfc8 3. Bxg7 Kxg7
>>>>>>>>>>>                                    4. Qf6+ Kf8 5. Bg6 Rc7 6. Rf1 Re8 7.
>>>>>>>>>>>                                    Qh8+ Ke7 8. Rxf7+ Kd8 9. Qxe8+ Kxe8
>>>>>>>>>>>                                    10. Rxc7+ Kd8 11. Rxb7 Qxc3 12. Rxa7
>>>>>>>>>>>                                    Qxe5
>>>>>>>>>>>               12     2:18     ++   1. ... a5!!
>>>>>>>>>>>               12     3:39  -2.57   1. ... a5 2. Bf6 Rfc8 3. Qg4 Qf8 4.
>>>>>>>>>>>                                    Qe4 g6 5. Kb2 Qc5 6. Rd1 Ra6 7. Qf4
>>>>>>>>>>>               12     4:07  -2.58   1. ... Rfb8 2. Qe4 g6 3. Rf1 Qf8 4.
>>>>>>>>>>>                                    Kb2 c5 5. Qg4 g5 6. Bf2 c4 7. Be2
>>>>>>>>>>>         (4)   12->   4:18  -2.58   1. ... Rfb8 2. Qe4 g6 3. Rf1 Qf8 4.
>>>>>>>>>>>                                    Kb2 c5 5. Qg4 g5 6. Bf2 c4 7. Be2
>>>>>>>>>>>         (3)   13     5:25  -2.52   1. ... Rfb8 2. Kb2 Rc8 3. Rf1 Rc7 4.
>>>>>>>>>>>                                    Qe4 g6 5. Rxf7 Rxf7 6. Qxg6+ Kf8 7.
>>>>>>>>>>>                                    Qxh6+ Kg8 8. Qg6+ Rg7 9. Qxe6+ Rf7
>>>>>>>>>>>               13     7:40  -2.54   1. ... Rfc8 2. Rf1 Qf8 3. Qe4 g6 4.
>>>>>>>>>>>                                    Kb2 Rc7 5. Bf6 c5 6. Qg4 c4 7. Be2
>>>>>>>>>>>                                    Rd7 8. h4
>>>>>>>>>>>               13     8:00  -2.55   1. ... Rfe8 2. Kb2 Rab8 3. Rd1 Qf8
>>>>>>>>>>>                                    4. Qe4 g6 5. Bf6 c5 6. Qg4 c4 7. Be2
>>>>>>>>>>>                                    Qc5
>>>>>>>>>>>         (3)   13->   8:00  -2.55   1. ... Rfe8 2. Kb2 Rab8 3. Rd1 Qf8
>>>>>>>>>>>                                    4. Qe4 g6 5. Bf6 c5 6. Qg4 c4 7. Be2
>>>>>>>>>>>                                    Qc5
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>I will have some 64-cpu alpha numbers in a month or two.  Working on a port
>>>>>>>>>>>to use UPC right now...  Compaq is loaning me a single-cpu alpha to compile/test
>>>>>>>>>>>on with the target of a 64 cpu machine they have.  I will try to get it on to
>>>>>>>>>>>ICC on a weekend maybe...  Or maybe for the next CCT.  :)
>>>>>>>>>>
>>>>>>>>>>Prophecy:
>>>>>>>>>>You will win the next WCCC[*]
>>>>>>>>>>
>>>>>>>>>>[*] Unless someone else does the same port.  There is no other machine that even
>>>>>>>>>>comes close.
>>>>>>>>>
>>>>>>>>>I am not so sure that it is enough to win.
>>>>>>>>>In the last 2 WCCC tournament the biggest hardware did not win.
>>>>>>>>>
>>>>>>>>>Deep thought failed to win in 1995(Fritz3 was the champion)
>>>>>>>>>Deep Junior,Deep Fritz,Ferret failed to win in 1999 and Shredder won.
>>>>>>>>>
>>>>>>>>>Uri
>>>>>>>>
>>>>>>>>
>>>>>>>>In the last two WCCC's there were no programs that were _really_ searching
>>>>>>>>at 60M nodes per second either.  :)
>>>>>>>
>>>>>>>Yes but in the WCCC of 1995 Fritz was also clearly slower and I also believe in
>>>>>>>diminishing returns so 2M against 60M is not the same as
>>>>>>>0.1M against 3M.
>>>>>>>
>>>>>>>Uri
>>>>>>
>>>>>>
>>>>>>I don't believe in "diminishing" returns when it is computer vs computer.  Give
>>>>>>me that extra ply _any_ day.  It will swing the match in my favor if my opponent
>>>>>>and I are equal at equal search depths.
>>>>>
>>>>>I believe in diminishing returns between different programs for the same reason
>>>>>that diminishing return may happen in comp-human games.
>>>>>
>>>>>At small depthes tactics dominates so the 30 times fastesr program usually wins.
>>>>>
>>>>>At big depthes there are things that one program understands and the second
>>>>>program does not understand when depth is not going to help.
>>>>>
>>>>>If 2 different programs have different positional weaknesses then the slower
>>>>>program has practical chances to win at big depthes.
>>>>>
>>>>>Uri
>>>>
>>>>
>>>>
>>>>OK... then at _today's_ computer speeds, I don't believe in diminishing
>>>>returns yet.  In 20 years, perhaps.  But the difference between a 15 ply
>>>>search and a 17 ply search is _significant_ still.  Lots of experiments have
>>>>shown that diminishing returns don't appear to happen at any depth we can
>>>>reach today, even using 24 hours of computer time.
>>>
>>>
>>>What about Ernst Heinz's fixed-depth, self-play matches with Fritz?  They seemed
>>>to strongly suggest diminishing returns, even at depths much shallower than 15
>>>or 17 plies.
>>>
>>>-Peter
>>
>>
>>Perhaps the program?  Hans Berliner did an interesting experiment a long while
>>back, and concluded that "dumber" programs show this diminishing return problem
>>sooner than "smarter" programs.
>
>I do not think that Fritz6 is dumb.
>A dumb program has no chance to lead the ssdf list that is no blitz.

There I totally disagree.  It is _clear_ that older versions of Fritz were
definitely "dumb" whether the current version is or not.  And _they_ were able
to hit the top of the SSDF as well.  To beat other computers is a different
thing than to compete with human GM players...



>
>The result that I saw in Ernst Heinz's book do not prove diminishing return but
>they suggest that diminishing returns happens and you simply do not have enough
>games to prove it.

It might be that when we can reach depths beyond 20 plies in a normal game,
this might begin to happen.  I don't know with no data to support it.  But
we aren't going to hit 20 plies for another two decades or so...




>
>I believe that the main problem is that the diminisihing return is not big
>enough and you need more games to prove it.



That is possible.  But if it is 20+ years off before it becomes a problem,
then I don't consider it an issue today...


>
>I agree that programs with less knowledge in the evaluation should show bigger
>diminishing returns but I still believe that for all programs there is a
>deminshing returns.


I believe there is life on other planets too.  We just don't have any proof
yet and probably won't for a long time.


>
>I believe that part of the deminishing returns is result of opening book but not
>all of them.
>I decided to try the nunn2 match to find if there is diminishing returns not
>because of opening book(I believe that part of the openings gives no practical
>chance to win for both sides if you search deep enough and I believe that the
>nunn2 position are not positions when it is a practical problem).
>
>I do not find diminishing returns but I did not use a lot of computer time for
>it and tested only depthes that are at most 9.
>
>  Ernst also concluded that for the time being,
>>at least thru 15-16 plies, there was no apparent 'diminishing returns' for his
>>program when he replicated the tests Monty and I did...
>
>These tests are not games and the only valid test is games.
>I guess that Ernst is going to find no apparant deminishing returns also for
>fritz6 by the tests that Monty and you did.
>
>Uri


I'd bet he would.  However, when you look at the data, and if you believe the
premise that says a search 1 ply deeper is more accurate than the previous
search, even if we don't agree by how much, then the experiment showed that
programs continue to change their minds at a near constant rate for many more
plies beyond today's hardware capability.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.