Author: Günther Simon
Date: 03:08:13 01/23/06
Go up one level in this thread
On January 23, 2006 at 03:01:16, Jonas Cohonas wrote: >Rank Engine Score % >1 Rybka v1.01 Beta 11.w32 138.5/220 62.9 · ······ >2 Shredder 9.1UCI 41.5/110 37.7 35-62-13 >3 Fruit2.2.1 40.0/110 36.3 26-56-28 > >I had set it to play 220 games based on the Rybka 220 opening positions (for >testing) .epd which means that they made it to position #54. > >For some odd reason they played 221 games and there where a few games ended due >to "illigal move" on the first move and it always seems to happen when the first It did not happen always in the first move! Sometimes it happened 10 moves or 15 moves later and sometimes it happened even in move 60 or 70. Also those are not a 'few' games. After eliminating 41! buggy illegal move results and a few buggy conflicting results, only 179 of 220 games were left. And even those still contained at least 2 time losses of Shredder despite an increment tc(180+2)? >move made was a knight move, if this is due to a bug in arena (Arena 1.99beta2), Have you started with beta 1 and later changed to beta2? I thought all those results bugs were fixed meanwhile? I wonder why all the last 45 or so games had no buggy results? Note also that the Arena site says those new betas are only for testing and shouldn't be used for games(posted somewhere). >the .epd positions, one of the engines i don't know, but after a quick scan it >seems evenly devided, however this makes the result a little bit unreliable, but A quick scan is not enough, if you want to post reliable results. Also, _if_ all the buggy results would have been 'evenly' divided (which they aren't), you still had to simply eliminate them. Evenly errors only won't change result outcomes between exactly evenly programs and this is only the case between identical programs. It is very easy and was often told here, because some people e.g. defended a very buggy book choice by the same sentence: 'The buggy lines were evenly divided', that's just nonsense, because at least one program (in a match) is handicapped if it is just marginally better than the other... The same here, buggy results/lines, whatever, cannot be 'evenly' divided. Please delete all that buggy results next time, before posting and releasing a pgn file. After deleting 41 buggy results (still including 2 fishy Shredder time losses) the outcome is: Rybka 118.5/179 66.20% Fruit 31.0/ 91 34.06% Shredder 29.5/ 88 33.52% >should reflect pretty well the difference in strength. > >Rybka played with default settings and so did both Fruit and Shredder. > >For more info and all the games zipped: > >http://www.fusionweb.dk/chess/rybka/220/ Guenther
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.