Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: PB-ON vs PB-OFF (final results)

Author: Ed Schröder

Date: 05:32:15 10/15/99

Go up one level in this thread


>Posted by Inmann Werner on October 15, 1999 at 06:32:41:
>
>>To enrich the discussion about the value of the "Permanent Brain" (PB) I
>>have started 2 experiments with Rebel Century (RC) which will give some
>>data for a better judgement of the value (in elo) of the permanent brain.
>>
>>Thinking it all over I still believe that the best way to provide data is
>>to test the SAME program (PB_ON) versus (PB_OFF).
>>
>>Arguments:
>>
>>#1. Both programs are equal in strength.
>>
>>#2. There is no single evidence (yet) that the % of predicted moves is
>>higher playing the same program. Rebel (or any other program) percentage
>>against Program_X might be 25% and against Program_Y 75% and against
>>itself maybe 50%, nobody knows.
>>
>>The final result:
>>
>>Experiment-1:
>>RC (PB=ON) vs RC (PB=OFF)
>>100 auto232 games
>>Time control: 60 secs average.
>>Hardware: 4xPII-266 + 2xPII-450
>>Result: 61-39
>>
>>Experiment-2:
>>RC (PB=ON) vs RC (PB=OFF)
>>100 auto232 games
>>Time control: RC (PB=ON) 30 sec average
>>Time control: RC (PB=OFF) 60 sec average
>>Hardware: 4xPII-266 + 2xPII-450
>>Status: 57-43
>>
>>I find the 57-43 result quite shocking. I expected (and hoped for) a close
>win
>>for the PB_ON version but not a clear victory.
>>
>>I think it would be a wise thing if others are willing to do the same
>>experiment with other programs. Somehow I have the feeling you will
>>totally different results doing so.
>>
>>Ed Schroder
>
>Experiment 2 is really interesting!
>
>PB=ON Engine has 30 secs + 60 secs ponder Time
>PB=OFF Engine hash 60 sec only.
>Have you any chance to get the percent of PB hits?

No idea.

>how fast (NPS) are the two machines compared?

On a PII-256 RC hits 90,000 average on the PII-450 that is 140,000.

>Do you think, that also if PB searches the wrong move, the filling of the
>hashtables is such extremly useful?

It would save 10%.

>How about the time management. Do both engines really use their time, they
>have, nearly equal?

Both engines use the same TC. Meaning to say that this favors the PB_ON
version because it has been programmed for that. The PB_OFF has a
big disadvantage because the time-control is not developed to play without
the PB when the topic is playing strength. This issue IMO is an important
KEY point in the discussion about the value of ENG-ENG (or Winboard)
matches.

>Much questions arise...., or is it only statistic, "bad luck" for the PB=OFF
>engine?

100 games is a strong indicator. Personally I use 200 games as a reference.
Most of the time the differences between 100 and 200 games are low (about 5%)
sometimes peaking to 10-15% but these are big exceptions. Above 200 games
big fluctuations are rare.

Ed

>Werner




This page took 0.02 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.