Author: blass uri
Date: 16:46:14 12/16/98
Go up one level in this thread
On December 16, 1998 at 16:27:49, Komputer Korner wrote: >On December 16, 1998 at 02:44:38, Christophe Theron wrote: > >>On December 15, 1998 at 23:41:50, Kevin Mulloy wrote: >> >>>Gentlemen, >>> I am in the process of testing MPC7 in a 200 MMX 80 meg RAM machine against >>>MCP7 in a 450PII 256 meg RAM machine. In the 200, the MCP7 engine calculates >>>aprox 1,250,000 positions per minute. In the 450, about 2,600,000 positions per >>>min (slightly more than double). I have adjusted the times accordingly -- I >>>give the 200 2 min per move and the 450 1 min per move. I thought that this >>>should be a fairly even match with a slight edge to the 450. After 20 games, >>>the 200 leads 15-5?? I was not testing this fine program against itself to try >>>to produce a "winning" machine. I just wanted an easy way to handicap one >>>machine so that the 200 and the 450 would play fairly even chess when I matched >>>different programs against Mcp7. I realize that I have a very small sample here >>>-- could someone advise me on the proper set up that I should be looking for to >>>even these 2 machines out. Also, If my approach to this problem is OK, how much >>>bigger should the sample be to give a good indication of strength? Thank you, >>>Kevin Mulloy >> >>My opinion is that you should not use an average time per move, but use a given >>time for the whole game, or classical time controls (A moves in B minutes, then >>C moves in D minutes, and so on). >> >>Depending on the program, average time per move is sometimes completely erratic. >>Maybe it is the case with MChess? >> >>Also, if you want such a match to be fair, you have to disable thinking on >>opponents time. The trouble here is that not doing so should have given an >>advantage to the fastest computer, so really I cannot explain your result. >> >>Strange... >> >> >> Christophe > >The problem of testing the same program against each other on 2 different >machines with different speeds (even if you handicap them fairly) is well known. >The only way it would be fair is if the program was a stict brute forcer with no >extensions and no pruning nor null move nor selective search of any kind. I think that the only way to do it fair is when you disable thinking at the opponent's time I do not think that it is relevant if the program use extensions or does not use extensions. >However all programs have some form of pruning so the test is biased. Usually >the bias would be for the faster machine if both had the same time control in >that the expected win of the faster machine would be more than proportional to >the speed advantage. In this case since you handicapped with a longer time >control the bias is towards the longer time control. I do not understand it. If the slower machine(with the long time control) does not look at more nodes per move then I do not understand why do you think that it has an advantage. Uri Unfortunately at short time >controls, there isn't a perfect correlation between search depth and time >control when measuring on different cpus. Hash tables are one example that >screws up your search depth handicapping. It makes me think that M-Chess is a >larger selective pruner than we have all thought. However your sample is way too >small to get any indication of bias. >-- >Komputer Korner
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.