Author: José de Jesús García Ruvalcaba
Date: 11:35:59 12/17/98
Go up one level in this thread
On December 16, 1998 at 19:46:14, blass uri wrote: > >On December 16, 1998 at 16:27:49, Komputer Korner wrote: > >>On December 16, 1998 at 02:44:38, Christophe Theron wrote: >> >>>On December 15, 1998 at 23:41:50, Kevin Mulloy wrote: >>> >>>>Gentlemen, >>>> I am in the process of testing MPC7 in a 200 MMX 80 meg RAM machine against >>>>MCP7 in a 450PII 256 meg RAM machine. In the 200, the MCP7 engine calculates >>>>aprox 1,250,000 positions per minute. In the 450, about 2,600,000 positions per >>>>min (slightly more than double). I have adjusted the times accordingly -- I >>>>give the 200 2 min per move and the 450 1 min per move. I thought that this >>>>should be a fairly even match with a slight edge to the 450. After 20 games, >>>>the 200 leads 15-5?? I was not testing this fine program against itself to try >>>>to produce a "winning" machine. I just wanted an easy way to handicap one >>>>machine so that the 200 and the 450 would play fairly even chess when I matched >>>>different programs against Mcp7. I realize that I have a very small sample here >>>>-- could someone advise me on the proper set up that I should be looking for to >>>>even these 2 machines out. Also, If my approach to this problem is OK, how much >>>>bigger should the sample be to give a good indication of strength? Thank you, >>>>Kevin Mulloy >>> >>>My opinion is that you should not use an average time per move, but use a given >>>time for the whole game, or classical time controls (A moves in B minutes, then >>>C moves in D minutes, and so on). >>> >>>Depending on the program, average time per move is sometimes completely erratic. >>>Maybe it is the case with MChess? >>> >>>Also, if you want such a match to be fair, you have to disable thinking on >>>opponents time. The trouble here is that not doing so should have given an >>>advantage to the fastest computer, so really I cannot explain your result. >>> >>>Strange... >>> >>> >>> Christophe >> >>The problem of testing the same program against each other on 2 different >>machines with different speeds (even if you handicap them fairly) is well known. >>The only way it would be fair is if the program was a stict brute forcer with no >>extensions and no pruning nor null move nor selective search of any kind. > >I think that the only way to do it fair is when you disable thinking at the >opponent's time > I do not like testing with pondering disabled (I know people will keep doing it for many reasons, I am only stating my opinion), because I see the whole program as a unit; and the time management policy is an integral part of the program: it can be a strength or a weakness. And I see that pondering is very relevant to the time management policy. >I do not think that it is relevant if the program use extensions or does not use >extensions. > >>However all programs have some form of pruning so the test is biased. Usually >>the bias would be for the faster machine if both had the same time control in >>that the expected win of the faster machine would be more than proportional to >>the speed advantage. In this case since you handicapped with a longer time >>control the bias is towards the longer time control. > >I do not understand it. >If the slower machine(with the long time control) does not look at more nodes >per move then I do not understand why do you think that it has an advantage. > >Uri > > Unfortunately at short time >>controls, there isn't a perfect correlation between search depth and time >>control when measuring on different cpus. Hash tables are one example that >>screws up your search depth handicapping. It makes me think that M-Chess is a >>larger selective pruner than we have all thought. However your sample is way too >>small to get any indication of bias. >>-- >>Komputer Korner
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.