Author: Mike Byrne
Date: 18:46:30 06/01/03
Go up one level in this thread
On June 01, 2003 at 16:18:56, Kurt Utzinger wrote: >On June 01, 2003 at 16:13:25, Jorge Pichard wrote: > >>Athlon XP 2800+ with 512 MB, both programs using 64 MB hashtable. >>After 9 games at G\90, both programs using Fritz8.ctg in Engine vs Engine match >>Abrok 5 is leading by one game! >> >>http://home.t-online.de/home/roman.korba/ > >You have still not played enough games. I give below an example of a match >[40'/40] I have played over 100 games between Gandalf 4.32g and Program_X [I am >a beta tester of X] to show what I mean: > >Gandalf 4.32g vs Program X > >Games 1-10 >3.0-7.0 [win program X] >Total 3.0-7.0 for program X > >Games 11-20 >6.5-3.5 [win Gandalf] >Total 9.5-10.5 for program X > >Games 21-30 >5.0-5.0 [draw] >Total 14.5-15.5 for program X > >Games 31-40 >3.5-6.5 [win program X] >Total 18.0-22.0 for program X > >Games 41-50 >4.5-5.5 [win program X] >Total 22.5-27.5 for program X > >Games 51-60 >3.0-7.0 [win program X >Total 25.5-34.5 for program X > >Games 61-70 >5.0-5.0 [draw] >Total 30.5-39.5 for program X > >Games 71-80 >8.0-2.0 [win Gandalf] >Total 38.5-41.5 for program X > >Games 81-90 >7.0-3.0 [win Gandalf] >Total 45.5-44.5 for Gandalf > >Games 91-100 >5.5-4.5 [win Gandalf] >Final match result 51.0-49.0 for Gandalf > >Can anybody tell me for sure which of the above two is the stronger program?? >And what about if I had only played a 20 games match and these games would have >been those played in rounds 71-90? Then, the result would have been 15.0-5.0 in >favour of Gandalf 4.32g!! Imagine what some testers would have argued about the >strenght of program X? > >For all these reasons I think that something concrete about the strength between >two programs can only be said if 100, better 200-300 games or even more have >been played. > >Kurt Kurt - that is so true - especially when engines are equal - when two engines are perfectly equal -- it's like red and black at the casino --how often do you see red or black dominate the roulette wheel for short period of time - 30 or so spins on the wheel - computer chess is the same when engines are equal - I think you need 500 games head to head games or so to determine the better program esp when they are close ....now we can understand why chess programers "hate" swiss tournament - there is even more chance added in to the equation - the best programs do not always win a swiss tournament -- 1995 Hong Kong comes to mind. Best, Michael
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.