Computer Chess Club Archives




Subject: Re: Dutch Open Leiden after round 6

Author: Uri Blass

Date: 08:08:47 10/21/02

Go up one level in this thread

On October 21, 2002 at 10:37:19, Vincent Diepeveen wrote:

>On October 21, 2002 at 09:43:57, Uri Blass wrote:
>>On October 21, 2002 at 08:05:54, Tony Werten wrote:
>>>On October 21, 2002 at 03:48:05, Jeroen Noomen wrote:
>>>>On October 21, 2002 at 02:27:01, Tony Werten wrote:
>>>>You don't know if your improvement is worth 200 elopoints,
>>>>therefore you need to test it thoroughly first :-)
>>>Test it ? You mean like play against other programs ? Then what have I been
>>>doing this weekend ?
>>1)If you have a real big improvement of 200 elo you can test it and get
>>conclusions in relatively a short time.
>That is not true. Books are too important nowadays. If a couple of
>hundreds of thousands of moves get added to a book, then you HAVE
>to test all the new lines or you get crushed.
>A jump of 200 points from the stronger engines, including strong
>amateurs, is not possible.
>However playing with a new beta version playing hundreds of points
>weaker, that IS possible.
>the biggest bugs always appear either in the first move out of book,
>or the far endgame. It is a known thing.
>also very well known bug is bug in time division.
>THAT you can test in a short period of time.
>But even despite that short period of time most use to test,
>including me (big big problem for diep therefore this weekend),
>such bugs somehow do not get out of engines.
>I remember insomniac icsvn2
>and i remember first few games of ZZZZZZ this tournament where it
>also forfeited on time because of using 2-3 minutes each move.
>In short, you HAVE to test well.
>Even your statement is PROVEN to be not true.
>Just a short period of testing, like 200 winboard blitz games and
>a few testsets is not enough. period.
>>The main problem is when you believe that the improvement is 20-30 elo.
>>In this case it is better to use an old version.
>Only if you start your engine it is possible to jump quickly from 1000 rating
>to 2400 rating. Above that, no 200 point improvements, unless you use
>a rating system i do not know yet.
>>If the idea how to improve your program by 200 elo is something that you
>>implemented only in the last minute(less than 24 hours before the tournament)
>>then it is better not to use it before you test it and delay the use of it for
>>the second part of the tournament when you have time to test
>>24 hours are enough to get 8 games at 90 minutes per game and if you get 6-2
>>result against a program at similiar level to your previous version then you can
>>be at least sure that the new version is not significantly weaker at the time
>>control of the tournament.
>8 games 90 0 is not enough to fix bugs in book, not to mention see bugs
>in some parts of the evaluation you 'fixed'.
>Some things in search, like move ordering, extension on or off, such things
>you can RISK doing last moment before tournament.
>I do not even take that risk however.
>Rewriting i/o i am busy with. i would take *that* risk for coming weekend,
>because it should still play the 100% same moves like the old versoin.
>so i only have to test whether it doesn't crash. that's all. Hard enough
>already to test within 5 days.
>>2)I have a question.
>>Is there a chance that we are going to see Xinix or Diep(you can answer only for
>>xinix) in the WBEC tournament of Leo.
>DIEP is not winboard anymore.
>DIEP has its own GUI. If leo wants it, he can buy it within some time
>and load his winboard engins within DIEP. It is however not released
>yet. Winboard can get loaded in the diep GUI.
>What ranking does Shredder which is UCI have in his league?
>>In the first division there are strong programs like
>>Vincent claimed that Diep beated Nimzo98 10-0.
>Yes but this is with a bit older tournament book that is well tested.
>I do not release diep's tournament book ever, just like Jeroen Noomen
>doesn't release the tournament book for tiger and just like Alexander
>Kure doesn't release the tournament book for  Fritz and just like
>alexander Kure doesn't release the tournament book of Nimzo.
>Even then i would expect major scores.
>I hope you realize if you get out of book with a pawn less, that
>winning is very hard.
>Nimzo2000 is completely different from nimzo98, because nimzo98 had
>a book from 1998 and nimzo98 would never get above 10 ply much.
>Nimzo2000 does and it has a book from 2000, which is like 2 years
>difference and a very important difference.
>I do not see why you feel that 2 years of difference is not 'significantly'
>better. What is that for stupid statement you made?
>The book is a major difference. The 2000 book from nimzo2000 has the
>NCO99 book already incorporated into it. that is a *major* step forward
>in openings theory compared to the old crap book from 1997 (crap in
>2002, not crap in 1997 of course).
>Or do you deny that?

I did not test the books of nimzo so I do not know but Nimzothis is from the
ssdf list

29 Nimzo 8.0  128MB K6-2 450 MHz          2555   22   -22   984   50%  2554
31 Nimzo 7.32 128MB K6-2 450 MHz          2550   21   -21  1070   54%  2519
43 Nimzo 99  128MB K6-2 450 MHz           2485   22   -22   996   44%  2527
52 Nimzo 99  64MB P200 MMX                2447   23   -23   885   51%  2440
54 Nimzo 98  58MB P200 MMX                2431   19   -19  1352   51%  2427

Nimzo99 is 16 elo better than Nimzo98 on p200
Nimzo8 is 70 elo better than Nimzo99 on K6-450.

Note that nimzo2000 is older than Nimzo7.32 and Nimzo8 based on my memory so
the total difference seems to be less than 86 elo.

This difference is not a difference of something close to 10-0

Even when the difference is 200 elo the ssdf usually does not get 10-0 results.

Here are the result of Nimzo98 in the ssdf

You can see that even with inferior hardware (p200 against A1200) it can get
more than 10% against programs of 2001 on A1200(tiger14,Deep Fritz,Junior7)

54 Nimzo 98  58MB P200 MMX, 2431
CT14.0 A1200     5.5-34.5  DpFritz A1200      3-17    Junior7 A1200      3-17
Shre532 A1200      3-17    GambTig K6450   15.5-24.5  Junior7 K6450    9.5-36.5
Shre532 K6450    8.5-31.5  Junior6 K6450      1-6     Nimzo 8 K6450   17.5-24.5
SOS  K6-2 450     18-28    Nimzo99 K6450     17-23    CM6000 P200X       5-5
Hiar732 P200X   18.5-21.5  Fritz 5 P200X   16.5-23.5  Hiarcs7 P200X     15-25
Nimzo99 P200X     20-22    Junior5 P200X   16.5-23.5  Rebel 9 P200X     22-18
Hiarcs6 P200X     22-18    Rebel 8 P200X   16.5-23.5  MCP 6 P200MMX   25.5-16.5
Zark5 P200MMX   10.5-9.5   Shred 2 P200X     18-22    MCP 7 P200MMX   34.5-25.5
MCP 8 P200MMX   23.5-16.5  Genius5 P200X     25-15    Gandal3 P200X      1-1
Kallis2 P200X   10.5-9.5   Rebel 9.0 P90     26-14    Hiarcs 6 P90    26.5-13.5
Genius 5 P90      17-4     MCPro 6.0 P90   22.5-17.5  Genius 4 P90    21.5-14.5
Nimzo 3.5 P90   28.5-11.5  Junior 4 P90      26-14    Geniu4 486/66   16.5-7.5
192  P200MMX    18.5-3.5   Kallis198 P90   33.5-6.5   Fritz3 486/66    7.5-0.5
SPARC 20 MHz      20-4     Comet32 P90       18-2


This page took 0.01 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.