Computer Chess Club Archives


Search

Terms

Messages

Subject: The Fritzmark is not reliable , use the Crafty Benchmark

Author: Drexel,Michael

Date: 11:28:03 04/09/03

Go up one level in this thread


On April 09, 2003 at 11:52:48, Charles Worthington wrote:

>I ran the Deepfritzmark and Shreddermark tests with hyperthreading disabled then
>enabled with some very confusing results that I am hoping someone can help
>explain:
>
>Test set #1 Hyperthreading Disabled, 64MB Hash, Engine Parameters @ default
>
>Shredder 7.04:  Shreddermark: 2227 +- 0 (1.5s) 705kN/s
>
>Deep Fritz 7 :  Deepfritzmark: 2724 +- 44 (3.1s)  2252kN/s
>
>
>
>
>Test set #2 Hyperthreading Enabled, 64MB Hash, Engine Parameters @ Default
>
>Shredder 7.04:  Shreddermark: 2227 +- 0 (1.5s) 803kN/s
>
>Deep Fritz 7 :  Deepfritzmark: 2476 +- 0 (3.2s) 2555kN/s
>
>
>
>Test set #3: Hyperthreading Enabled, 32 MB Hash, Engine Parameters @ Default
>
>Shredder 7.04: Shreddermark: 2784 +- 0 (0.4s) 907kN/s
>
>Deep Fritz 7:  Deepfritzmark: 2476 +- 0 (3.4s) 2532kN/s
>
>
>
>Test set #4; Hyperthreading enabled, 16MB Hash, engine parameters @ default
>
>Shredder 7.04;  Shreddermark: 2784 +- 0 (0.4s) 1008kN/s
>
>Deep fritz 7:  Deepfritzmark: 2476 +- 0 (4.5s) 2544 kN/s
>
>
>
>This is somewhat confusing as Fritz scored the highest fritzmark with
>hyperthreading_disabled_ even though his kN/s were_far_lower. Shredder scored
>far better with it_enabled_ both in result, speed, and time to solution.
>Also Shredder seemed to benefit more from the smaller hash sizes where Fritz
>seemed relatively worsened by them. Does anyone have any insight as to these
>seemingly contradictory results? And would I be better to run Deep Fritz with
>the hyperthreading diasabled even though his kN/s is considerably lower?
>
>Charles

Im not surprised.

You can get a lower Fritzmark although your Computer is faster and has higher
kN/s values. The Fritzmark is not linear.
This seems to be complete Nonsense.

In addition they changed the Fritzmark ratings arbitrarily with the release of
Fritz 7 because the kN/s were smaller in the new versions of Fritz.

You should use the Crafty Benchmark because the results are far more reliable.
A Crafty Benchmark is for example used in the famous SPEC CPU 2000 benchmark.

http://www.specbench.org/osg/cpu2000/CINT2000/


This seems to be the only important Testposition for Fritzmark:

[D]1r5r/3b1pk1/p2p1np1/p1qPp3/2N1PbP1/2P2PN1/1PB1Q1K1/R3R3 b - - 0 1

The time it takes till the engine reaches the end of ply 10 is measured.

When you start the Test three positions are evaluated up to ply 6:

[D] 1q2r3/2p1bpkp/5np1/8/5B1P/5QNK/3P1PP1/R7 w - - 0 1

[D] r1b2rk1/1p2bppp/p4n2/4B3/1qBR4/2N5/P3QPPP/R5K1 w - - 0 1

[D] r4rk1/5p2/p1b1pQpq/8/1B2P3/2NR4/PPP3PP/1K6 w - - 0 1

Michael



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.