Author: Jonas Cohonas
Date: 05:06:49 01/23/06
Go up one level in this thread
On January 23, 2006 at 06:08:13, Günther Simon wrote: >On January 23, 2006 at 03:01:16, Jonas Cohonas wrote: > >>Rank Engine Score % >>1 Rybka v1.01 Beta 11.w32 138.5/220 62.9 · ······ >>2 Shredder 9.1UCI 41.5/110 37.7 35-62-13 >>3 Fruit2.2.1 40.0/110 36.3 26-56-28 >> >>I had set it to play 220 games based on the Rybka 220 opening positions (for >>testing) .epd which means that they made it to position #54. >> >>For some odd reason they played 221 games and there where a few games ended due >>to "illigal move" on the first move and it always seems to happen when the first > >It did not happen always in the first move! Sometimes it happened 10 moves or 15 >moves later and sometimes it happened even in move 60 or 70. >Also those are not a 'few' games. After eliminating 41! buggy illegal move >results and a few buggy conflicting results, only 179 of 220 games were left. >And even those still contained at least 2 time losses of Shredder despite >an increment tc(180+2)? Yeah that is odd indeed. >>move made was a knight move, if this is due to a bug in arena (Arena 1.99beta2), > >Have you started with beta 1 and later changed to beta2? >I thought all those results bugs were fixed meanwhile? I wonder >why all the last 45 or so games had no buggy results? >Note also that the Arena site says those new betas are only >for testing and shouldn't be used for games(posted somewhere). Nope all in the same version. >>the .epd positions, one of the engines i don't know, but after a quick scan it >>seems evenly devided, however this makes the result a little bit unreliable, but > >A quick scan is not enough, if you want to post reliable results. >Also, _if_ all the buggy results would have been 'evenly' divided >(which they aren't), you still had to simply eliminate them. >Evenly errors only won't change result outcomes between exactly evenly >programs and this is only the case between identical programs. >It is very easy and was often told here, because some people e.g. >defended a very buggy book choice by the same sentence: >'The buggy lines were evenly divided', that's just nonsense, because >at least one program (in a match) is handicapped if it is just >marginally better than the other... >The same here, buggy results/lines, whatever, cannot be 'evenly' divided. >Please delete all that buggy results next time, before posting and >releasing a pgn file. I wanted to release the whole .pgn as it contained errors ans someone (like you) could clear up the whole mess. >After deleting 41 buggy results (still including 2 fishy Shredder >time losses) the outcome is: > >Rybka 118.5/179 66.20% >Fruit 31.0/ 91 34.06% >Shredder 29.5/ 88 33.52% Well that is pretty damn close to the buggy result :) >>should reflect pretty well the difference in strength. >> >>Rybka played with default settings and so did both Fruit and Shredder. >> >>For more info and all the games zipped: >> >>http://www.fusionweb.dk/chess/rybka/220/ > >Guenther Thanks for cleaning the pgn file, any chance you could send me the cleaned up pgn file?
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.