Author: blass uri
Date: 13:14:39 12/17/98
Go up one level in this thread
On December 17, 1998 at 14:35:59, José de Jesús García Ruvalcaba wrote: >On December 16, 1998 at 19:46:14, blass uri wrote: > >> >>On December 16, 1998 at 16:27:49, Komputer Korner wrote: >> >>>On December 16, 1998 at 02:44:38, Christophe Theron wrote: >>> >>>>On December 15, 1998 at 23:41:50, Kevin Mulloy wrote: >>>> >>>>>Gentlemen, >>>>> I am in the process of testing MPC7 in a 200 MMX 80 meg RAM machine against >>>>>MCP7 in a 450PII 256 meg RAM machine. In the 200, the MCP7 engine calculates >>>>>aprox 1,250,000 positions per minute. In the 450, about 2,600,000 positions per >>>>>min (slightly more than double). I have adjusted the times accordingly -- I >>>>>give the 200 2 min per move and the 450 1 min per move. I thought that this >>>>>should be a fairly even match with a slight edge to the 450. After 20 games, >>>>>the 200 leads 15-5?? I was not testing this fine program against itself to try >>>>>to produce a "winning" machine. I just wanted an easy way to handicap one >>>>>machine so that the 200 and the 450 would play fairly even chess when I matched >>>>>different programs against Mcp7. I realize that I have a very small sample here >>>>>-- could someone advise me on the proper set up that I should be looking for to >>>>>even these 2 machines out. Also, If my approach to this problem is OK, how much >>>>>bigger should the sample be to give a good indication of strength? Thank you, >>>>>Kevin Mulloy >>>> >>>>My opinion is that you should not use an average time per move, but use a given >>>>time for the whole game, or classical time controls (A moves in B minutes, then >>>>C moves in D minutes, and so on). >>>> >>>>Depending on the program, average time per move is sometimes completely erratic. >>>>Maybe it is the case with MChess? >>>> >>>>Also, if you want such a match to be fair, you have to disable thinking on >>>>opponents time. The trouble here is that not doing so should have given an >>>>advantage to the fastest computer, so really I cannot explain your result. >>>> >>>>Strange... >>>> >>>> >>>> Christophe >>> >>>The problem of testing the same program against each other on 2 different >>>machines with different speeds (even if you handicap them fairly) is well known. >>>The only way it would be fair is if the program was a stict brute forcer with no >>>extensions and no pruning nor null move nor selective search of any kind. >> >>I think that the only way to do it fair is when you disable thinking at the >>opponent's time >> > > I do not like testing with pondering disabled (I know people will keep doing it >for many reasons, I am only stating my opinion), because I see the whole program >as a unit; and the time management policy is an integral part of the program: it >can be a strength or a weakness. And I see that pondering is very relevant to >the time management policy. I understand your point. I think that it is possible to do a fair test in another way if you have unequal hardware You can do the fast machine slower by running another program at the same time. The only problem is that it is not simple to slow the program by the right factor. Uri
This page took 0.23 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.