Author: Albert Silver
Date: 18:02:03 08/19/02
Go up one level in this thread
On August 18, 2002 at 20:54:47, Vincent Diepeveen wrote:
>On August 18, 2002 at 19:53:18, Albert Silver wrote:
>
>it is a bad idea to design a testset for 10-15 seconds now
From the response you both gave, I can see what I proposed was not well
understood. Think of it this way: suppose you are running a blunder check at 15
seconds a move. It notices that at move 27, with the engine at a mere 9-10 plies
or so, the eval changed strongly. It didn't see anything at move 26. Of course
this is because it is not thinking long enough, but you already know enough:
there was an error done somewhere around move 22-27 probably. If it had been
running at 3 minutes a move, it would have seen the problem at move 23. Now
suppose the game goes 40 moves deep before the losing side resigns. If you do it
at 3 minutes a move that means 80 plies or 240 minutes. In other words it will
take 4 hours. If I take 15 seconds a move it is done in one hour, and I can MOST
certainly find out where the blunder was in less than those extra 3 hours by
manually checking the 5 moves I presume is where it happened.
Albert
>because by the time the testset is finished movei is improved 100
>times and will find everything under 1 second, not to mention the
>already bugfixed other 100 engines. Also hardware increases, within
>6 months when the new intel hardware and new SGI machines (with mckinleys)
>get on the market it is completely outdated.
>
>nevertheless the collection in itself is a 1000000x better idea than what
>the 'wmtest' guys did. i obviously love to receive the uri blass testset.
>
>>On August 18, 2002 at 06:27:08, Uri Blass wrote:
>>
>>>On August 17, 2002 at 23:14:12, Vincent Diepeveen wrote:
>>>
>>>>On August 17, 2002 at 17:43:15, Mike S. wrote:
>>>>
>>>>i hope you realized i put serious time in some positions
>>>>of their testset, in order
>>>>to find out that i have put more time in the positions than
>>>>they have when i started emailing them about it.
>>>>
>>>>If they use the word time. they mean 'computer time'.
>>>
>>>I think that only computer time is needed to generate a good test suite.
>>>I try to generate a good test suite based on games of movei in the 3th division
>>>of the winboard programs but it is going to take time.
>>>
>>>I am going to put only mistakes when there is a difference of more than one pawn
>>>between the score of yace after the move and the score of yace before the move
>>>after long analysis.
>>
>>I don't think you need long analysis for this. Basically, if the eval drops
>>significantly even after short analysis (10-15 seconds per move) you can be sure
>>something happened either the last move or a few moves ago (maybe it only saw
>>the problem a few moves after the blunder). When you see this sudden change, you
>>can use your own judgement, with the help of an engine, to see what happened.
>>You could then determine what the best move should have been, and whether it
>>constitutes a good test move.
>>
>> Albert
>>
>>>
>>>I will be happy if there are people who are interested in helping me.
>>>
>>>Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.