Author: Uri Blass
Date: 08:08:47 10/21/02
Go up one level in this thread
On October 21, 2002 at 10:37:19, Vincent Diepeveen wrote: >On October 21, 2002 at 09:43:57, Uri Blass wrote: > >>On October 21, 2002 at 08:05:54, Tony Werten wrote: >> >>>On October 21, 2002 at 03:48:05, Jeroen Noomen wrote: >>> >>>>On October 21, 2002 at 02:27:01, Tony Werten wrote: >>>> >>>>You don't know if your improvement is worth 200 elopoints, >>>>therefore you need to test it thoroughly first :-) >>> >>>Test it ? You mean like play against other programs ? Then what have I been >>>doing this weekend ? >>> >>>Tony >> >>1)If you have a real big improvement of 200 elo you can test it and get >>conclusions in relatively a short time. > >That is not true. Books are too important nowadays. If a couple of >hundreds of thousands of moves get added to a book, then you HAVE >to test all the new lines or you get crushed. > >A jump of 200 points from the stronger engines, including strong >amateurs, is not possible. > >However playing with a new beta version playing hundreds of points >weaker, that IS possible. > >the biggest bugs always appear either in the first move out of book, >or the far endgame. It is a known thing. > >also very well known bug is bug in time division. > >THAT you can test in a short period of time. > >But even despite that short period of time most use to test, >including me (big big problem for diep therefore this weekend), >such bugs somehow do not get out of engines. > >I remember insomniac icsvn2 >and i remember first few games of ZZZZZZ this tournament where it >also forfeited on time because of using 2-3 minutes each move. > >In short, you HAVE to test well. > >Even your statement is PROVEN to be not true. > >Just a short period of testing, like 200 winboard blitz games and >a few testsets is not enough. period. > >>The main problem is when you believe that the improvement is 20-30 elo. >>In this case it is better to use an old version. > >Only if you start your engine it is possible to jump quickly from 1000 rating >to 2400 rating. Above that, no 200 point improvements, unless you use >a rating system i do not know yet. > >>If the idea how to improve your program by 200 elo is something that you >>implemented only in the last minute(less than 24 hours before the tournament) >>then it is better not to use it before you test it and delay the use of it for >>the second part of the tournament when you have time to test > >>24 hours are enough to get 8 games at 90 minutes per game and if you get 6-2 >>result against a program at similiar level to your previous version then you can >>be at least sure that the new version is not significantly weaker at the time >>control of the tournament. > >8 games 90 0 is not enough to fix bugs in book, not to mention see bugs >in some parts of the evaluation you 'fixed'. > >Some things in search, like move ordering, extension on or off, such things >you can RISK doing last moment before tournament. > >I do not even take that risk however. > >Rewriting i/o i am busy with. i would take *that* risk for coming weekend, >because it should still play the 100% same moves like the old versoin. > >so i only have to test whether it doesn't crash. that's all. Hard enough >already to test within 5 days. > >> >>2)I have a question. >> >>Is there a chance that we are going to see Xinix or Diep(you can answer only for >>xinix) in the WBEC tournament of Leo. > >DIEP is not winboard anymore. > >DIEP has its own GUI. If leo wants it, he can buy it within some time >and load his winboard engins within DIEP. It is however not released >yet. Winboard can get loaded in the diep GUI. > >What ranking does Shredder which is UCI have in his league? > >>In the first division there are strong programs like >>Ruffian,Gandalf,Yace,Nimzo2000 > >>Vincent claimed that Diep beated Nimzo98 10-0. > >Yes but this is with a bit older tournament book that is well tested. > >I do not release diep's tournament book ever, just like Jeroen Noomen >doesn't release the tournament book for tiger and just like Alexander >Kure doesn't release the tournament book for Fritz and just like >alexander Kure doesn't release the tournament book of Nimzo. > >Even then i would expect major scores. > >I hope you realize if you get out of book with a pawn less, that >winning is very hard. > >Nimzo2000 is completely different from nimzo98, because nimzo98 had >a book from 1998 and nimzo98 would never get above 10 ply much. > >Nimzo2000 does and it has a book from 2000, which is like 2 years >difference and a very important difference. > >I do not see why you feel that 2 years of difference is not 'significantly' >better. What is that for stupid statement you made? > >The book is a major difference. The 2000 book from nimzo2000 has the >NCO99 book already incorporated into it. that is a *major* step forward >in openings theory compared to the old crap book from 1997 (crap in >2002, not crap in 1997 of course). > >Or do you deny that? I did not test the books of nimzo so I do not know but Nimzothis is from the ssdf list 29 Nimzo 8.0 128MB K6-2 450 MHz 2555 22 -22 984 50% 2554 31 Nimzo 7.32 128MB K6-2 450 MHz 2550 21 -21 1070 54% 2519 43 Nimzo 99 128MB K6-2 450 MHz 2485 22 -22 996 44% 2527 52 Nimzo 99 64MB P200 MMX 2447 23 -23 885 51% 2440 54 Nimzo 98 58MB P200 MMX 2431 19 -19 1352 51% 2427 Nimzo99 is 16 elo better than Nimzo98 on p200 Nimzo8 is 70 elo better than Nimzo99 on K6-450. Note that nimzo2000 is older than Nimzo7.32 and Nimzo8 based on my memory so the total difference seems to be less than 86 elo. This difference is not a difference of something close to 10-0 Even when the difference is 200 elo the ssdf usually does not get 10-0 results. Here are the result of Nimzo98 in the ssdf You can see that even with inferior hardware (p200 against A1200) it can get more than 10% against programs of 2001 on A1200(tiger14,Deep Fritz,Junior7) 54 Nimzo 98 58MB P200 MMX, 2431 CT14.0 A1200 5.5-34.5 DpFritz A1200 3-17 Junior7 A1200 3-17 Shre532 A1200 3-17 GambTig K6450 15.5-24.5 Junior7 K6450 9.5-36.5 Shre532 K6450 8.5-31.5 Junior6 K6450 1-6 Nimzo 8 K6450 17.5-24.5 SOS K6-2 450 18-28 Nimzo99 K6450 17-23 CM6000 P200X 5-5 Hiar732 P200X 18.5-21.5 Fritz 5 P200X 16.5-23.5 Hiarcs7 P200X 15-25 Nimzo99 P200X 20-22 Junior5 P200X 16.5-23.5 Rebel 9 P200X 22-18 Hiarcs6 P200X 22-18 Rebel 8 P200X 16.5-23.5 MCP 6 P200MMX 25.5-16.5 Zark5 P200MMX 10.5-9.5 Shred 2 P200X 18-22 MCP 7 P200MMX 34.5-25.5 MCP 8 P200MMX 23.5-16.5 Genius5 P200X 25-15 Gandal3 P200X 1-1 Kallis2 P200X 10.5-9.5 Rebel 9.0 P90 26-14 Hiarcs 6 P90 26.5-13.5 Genius 5 P90 17-4 MCPro 6.0 P90 22.5-17.5 Genius 4 P90 21.5-14.5 Nimzo 3.5 P90 28.5-11.5 Junior 4 P90 26-14 Geniu4 486/66 16.5-7.5 192 P200MMX 18.5-3.5 Kallis198 P90 33.5-6.5 Fritz3 486/66 7.5-0.5 SPARC 20 MHz 20-4 Comet32 P90 18-2 Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.