Author: Roger D Davis
Date: 22:06:19 01/08/03
Go up one level in this thread
On January 08, 2003 at 15:40:21, Rajen Gupta wrote: >On January 08, 2003 at 15:18:18, Roger D Davis wrote: > >>We occasionally see posts where engines take turns playing the Nunn positions, >>the idea being to give the engines an equal footing, but with some development. >>But has anyone looked at whether white or black wins more often for each Nunn >>position? Presumably, if white wins some 55% of the time for all openings (or >>whatever the figure it, consult your database), then the same should hold true >>for the Nunn positions: Given equal engines, white should win 55% of the time. >> >>Some large inequality between white or black would indicate that a particular >>position ought to be deleted from the set, or some other position substituted. >> >>Roger > >i see the point that you are trying to make ie that there maybe some positions >that are not strictly neutral to begin with; however each engine gets an equal >bite at the cherry so to speak and even if a particular position were to >inherently favour one colour, both engines get to play that colour and the final >result depends upon which engine can win more often; so i guess the nunn >positions are a pretty good test for evaluating engines; however the valid >argument is that programmers can tune their engines to perfom well on these >tests; Yes, each side gets an equal bite at the cherry, since the programs switch sides. But if this is all there is to it, then I might as well pick my own positions, and say that my suite is as good as Nunn's. My point: The defining characteristic of the Nunn suite is that the stronger engine can be expected to win approximately half the time (or 55% as white and 45% as black, assuming the initiative is worth 5%, an approximation). Accordingly, it becomes necessary to validate the positions themselves. Consider: Let's say we have a set of positions that favor white 90% of the time, and two development engines we want to compare. Neither has a good book yet. So...we run the engines through the suite, and find that with the white pieces, each engine wins 90% of the time, because the characteristics of the positions swamp any differences between the engines. Lesson: As the positions approach equality, finer differences in the strength of the engines can be revealed. So there is really something to say for validating the positions. Roger
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.