Author: Mike S.
Date: 13:50:27 09/06/00
Go up one level in this thread
On September 06, 2000 at 16:17:41, Uri Blass wrote: >(...) >They do not test the sucess in games but only success in test suites. >(...) >I think that the biggest problems are: > >1)The fact that in test suites there is only one good reply when in practical >cases there is more than one good reply. >There is usually one bad move that turned out to be good in test suites and it >is not the case in games. > >2)The fact that the test positions were composed by humans and there are many >cases when humans do not know the right move and these positions are not part >of a test suite. I agree: Test suit results cannot replace a judgement based on games, preferable many games against different opponents, etc. But I still think that test suites, especially if they contain many well chosen positions, are very useful to compare chess programs. They may not show their complete strength (you have mentioned examples of what isn't tested). But major components are visible more clearly even, than in games, i.e. you would have to analyse many games to find a really good remarkable combination. The *potential*, the *calculation abilities*, *endgame knowledge* etc. of chess programs can be compared more quickly and easily with a good test suite, than by 100 games IMO. A test position usually represents a kind of "peak of the game" situation, where a difficult move can decide the outcome. Of course some "gameplay" abilities to reach such a situation, are not tested in those test suites - But if the program just can't find the difficult move, than the previous efforts were pretty useless anyway! Furthermore, such situations can arise also, when the opponent blunders (talking of blunders which are not that obvious of course). The test shows, if the program could make use of it or not. Regards, M.Scheidl
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.