Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Reproducibility of Nunn matches is not so good

Author: Pete Galati

Date: 23:54:51 05/01/00

Go up one level in this thread


On May 02, 2000 at 02:03:12, Jouni Uski wrote:

>On May 02, 2000 at 01:54:47, Pete Galati wrote:
>
>>On May 02, 2000 at 01:10:58, Jouni Uski wrote:
>>
>>>About one week ago I posted "sensational" blitz Nunn test result Crafty17.10 -
>>>Fritz6a 12-8. And I wasn't lying! Now I have more time to re-check and second
>>>match ended 9 - 11 for Fritz. I also repeated two other matches with interesting
>>>results:
>>>                           1.              2.
>>>Crafty 17.10 - Lg2000   13 - 7         9,5 - 10,5
>>>Fritz6a - Lg2000        13,5 - 6,5     11 - 9
>>>
>>>Exact enviroment: AMD 450Mhz, ponder of, 16+16MB hash, 4m+1s level, 3+4+some
>>>5 piece TBs, Fritz6 interface, early resign.
>>>
>>>Conclusion: after 20 games You don't know much yet...
>>>
>>>Jouni
>>
>>I think your conclusion is correct, 20 games wouldn't be enough.  If I'm reading
>> your results the right way, Little Goliath won more games against both of it's
>>opponents in the second tests.  Does this posibly mean that LGoliath learns
>>better than Crafty and Fritz?  I don't know, I'm only asking.  Or again, is it
>>too few games to tell?
>>
>>Pete
>
>I test with LG2000 (and other engines) with learn off always.
>
>Jouni

Ok, make sense.  But on the other hand, does this sort of make sense?  Add the
results of both the first and second tests together, so that instead of having
20 test games, you have a a total of 40 test games.  Now you'll see that Crafty
is still ahead of LGoliath but by a smaller ratio, and that's probably closer to
being the actual reality.  Because in the first test, if you devide the Crafty
wins by the LGol wins, I think you got a number of approx 1.8(?) but is you
averaged the 2 tests, it comes out closer to 1.3.  I think 1.3 is much more
realistic because I doubt if Crafty is _that_ much better than LGol.

Pete



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.