Author: Thorsten Czub
Date: 03:44:42 04/05/02
Go up one level in this thread
On April 05, 2002 at 06:05:26, Uri Blass wrote: >It is possible to find which engine is better at blitz by >playing blitz games and if it means nothing then it means also >that 40/120 games meant nothing some years ago. sorry uri. but this is stupid. the programmers tune their versions on 40/120, because the ssdf guys test on 40/120. and the programmers try to tune on state of the art hardware. if you would tune on very old hardware, and present your program to the public, you would not know how it would behave. in the same way cars get tested. the people do NOT drive only 3 metres with the cars, because short way would be enough. they test normal speed, high speed, long distances, good tracks, bad ground, and and and. but nobody would test a car by driving only 3 metres. >One position is not a proof that the program is stronger. >A program may be better in position A and worse in position B. right. but i have played many many tournament games so far. and from all opponents, shredder6 paderborn was the heaviest enemy. shredder6 (cd-version) was weaker. all those games were on 400 mhz (40/120) or 1200 mhz (40/120). but still more serious than blitz games. >It may be possible to guess a small improvement based on test >of many positions but it is only a guess. > >evaluation may be changed based on the history of the game >(I know that it is at least the case for a previous version of >rebel) >so using only test positions when the engine has not the history >of the games is not the right way >to compare the strength of engines. and the right way to compare the strength of a chess engine is to play 1000 of blitz games ?? how funny this is. when we should value the quality of a movie, we should see 3 seconds extracts to find out. when we find out about people, we should interview them 30 seconds. when we want to find out about the quality of a book we should read 1 page of it, and when we want to find out about the quality of a city or a place to live we should watch a few pictures of the town ? blitz is a very short range. when i want to value a movie, i watch the whole movie. this takes as long as thze movie is, 90 minutes or more. when i want to find out about a car, i drive it many times, and this takes a while. when i want to learn about a human beeing, i do not only interview him 30 seconds. blitz games make fun. but they do not help much to find out about versions. a day has 24 hours. humans are used to value things within the range of a day or many days. 24 x 60 x 60 seconds. = 86400 seconds. you want to measure a chess program by let it play 300 seconds of chess ? thats 1/288 of a day !! humans are not used to measure something within 5 minutes. and even when you play 1000 games. all you do is counting the results but not the content of the games. > >Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.