Author: Christophe Theron
Date: 12:28:34 01/28/01
Go up one level in this thread
On January 28, 2001 at 14:33:23, Jorge Pichard wrote:
>Six Weeks ago I matched Nimzo 8 Vs Junior 6 at G\60 using an AMD Athlon 800 MHz
>And the score after 75 games ended up with a slight advantage of 6.5 points for
>Nimzo 8.
If I understand correctly (you should give complete results, it would be easier
to understand), this first match did not have enough statistical relevance.
> Then one week after that match, I decided to match the same two
>programs, but I used an AMD K6-2 500 MHz instead, and the score after 75 games
>ended this time in favor of Junior 6 by almost the same margin as the previous
>match.
Same here. The match did not have a significant result.
> I realized that certain program benefit more than others as the speed of
>the processor increases,
You cannot realize this from your data. You are just assuming it.
This assumption is in my opinion wrong, and has never been seriously
demonstrated. Your matches do not demonstrate it either.
It is very strange. Chess programs are all using, more or less, the same basic
principles. So the logical way is to assume that they all benefit more or less
equally from faster hardware. But I have never seen anybody supporting this
assumption. Instead of trying to demonstrate that this simple assumption is
wrong, everybody just assumes that it is wrong. Why?
I guess the answer is that it is more fun to assume that all chess programs do
not benefit from faster hardware in the same way. So people believe that by
changing the hardware or the time controls big surprises can happen...
On the other hand it is always hard to explain that in short matches big
surprises can happen FOR NO REASON.
So people tend to draw flawed conclusions based mainly on their beliefs, and to
present them as scientifical evidence...
> but I wasn't satified yet and decided to test the same
>two programs one more time with a slower processor this time. I asked my friend
>John to test these two programs again with his old Pentium Celeron 333 MHz at
>G\60 and after 9 games, Junior 6 was beating Nimzo 8 by W7 D2 L0 and, at that
>moment I decided to stop the match.
Wrong decision as this match has even less significance than the two others!
> I can only conclude that Nimzo 8 benefit
>more as the speed of the processor increase,
Wrong conclusion. Why don't you first conclude that your tests did not have
enough significance?
> therefore, Nimzo 8 will not have a
>great SSDF rating by the middle of February, but if you have a P.C. with a
>processors higher than 800 MHz Nimzo 8 is one of the few programs that benefit
>the most by using the latest technology available.
Once again, this has been said for virtually any program over the years and has
never been demonstrated.
Your matches do not demonstrate it. It is very important to achieve some
statistical reliability before you can draw any conclusion of this kind.
I would suggest that you play longer matches. I suggest at least 200 games per
match. Do not stop a match until it is over. Then report the results and we will
analyze them from a statistical point of view and we will see if they
demonstrate your point.
Christophe
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.