Author: Christophe Theron
Date: 11:49:01 10/18/01
Go up one level in this thread
On October 18, 2001 at 07:22:26, José Carlos wrote:
>On October 17, 2001 at 15:28:30, Christophe Theron wrote:
>
>>On October 17, 2001 at 14:56:16, Chris Taylor wrote:
>>
>>>I have played Fritz 3.10 against newish progams.
>>>F3 ran on an AMD 1200 inside F6 gui, using General.ctg
>>>1 x 1200 played against the PIII 733, the other played against the AMD 800
>>>The opponants
>>>
>>>PIII 733 Gambit Tiger 2.0, WcraftyP3 1811. This was offered as optimised for
>>>P3...
>>>Tiger ran with its own book, in F6 gui. Wcrafty's book is from Bob's site.
>>>Crafty ran under Remi Coulom's wbenging0047, with full auto232.
>>>
>>>AMD 800 Junior 4.6, Junior 6. Both versions of Junior ran under F6 gui, with
>>>Junior.ctg
>>>
>>>
>>>One of the oldest programs, that can run on my newest machines. Versus some
>>>newish stuff.
>>>
>>>Game in 1 hour...
>>>
>>>Junior 4.6 v Fritz 3.10 4-2
>>>Junior +3 -1 =2
>>>
>>>Junior 6 v Fritz 3.10 4½-1½
>>>Junior +3 -0 =3
>>>
>>>Gambit Tiger 2.0 v Fritz 3.10 5½-½
>>>Tiger +5½ -0 =1
>>>
>>>WcraftyP3 1811 v Fritz 3.10 4-2
>>>Crafty +3 -1 =2
>>>
>>>Just a small sample of games. Anyone wanting the pgn file of 24 games is
>>>welcome to it.
>>>
>>>Chris Taylor
>>
>>
>>
>>Is anybody still wondering if there have been progress in chess programming in
>>the last years?
>>
>>
>>
>> Christophe
>
> I don't say there isn't. It'd be absurd. But a couple of remarks:
>
> a. Your statement seems to imply that such a small number of games proves "the
>progress". I guess I got you wrong because you always claim a lot of games are
>needed to make any conclusion.
The overall result of the match is 18-6, that is a 75% win for the newer
software on inferior hardware.
I look at my statistical tables and I see that this result is significant with
90% confidence. The error margin for 24 games is under 15% with 90% confidence,
and here we got a result of 25% above 50%.
So I'm 90% sure that the new software on slower hardware is better than Fritz3
on the faster hardware.
I could even say that new software on slower hardware is *significantly* better
than Fritz3 on faster hardware, not even talking about what would happen on
equal hardware.
It's not as simple as "a lot of games are needed to make any conclusion".
It all depends on the number of games and the difference between the opponents.
When the difference in elo is big (as it is the case here), you do not need a
huge number of games.
However when the difference is small, you need LOTS of games.
I have already published several times the statistical tables that I am using,
but apparently nobody has any interest for this...
> b. If by "progress in chess programming" you mean "software-only progress"
Yes I mean "software only".
>(whatever that means -that concept is beyond my understanding, because I always
>optimize my code for a certain kind of hardware-), no conclusion can be made
>without testing in both new and old hardware. For example, if Fritz 3 is to be
>evaluated, 486-33 would be a good hardware to test the programs in. Then,
>comparing results in both kind of hardware would yield more interesting
>conclusions.
You overestimate what can be achieved by optimizing for a given hardware.
We are talking about a difference of 5% in speed here, depending on what
processor you have optimized for, and this speed difference corresponds to a 5
elo points difference. This is negligible because you would need a huge number
of games to detect such a difference.
Christophe
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.