Author: Dann Corbit
Date: 13:18:16 03/24/04
Go up one level in this thread
On March 24, 2004 at 15:30:32, Uri Blass wrote: >On March 24, 2004 at 14:32:01, Dann Corbit wrote: > >>On March 24, 2004 at 02:11:34, Uri Blass wrote: >> >>>On March 23, 2004 at 21:28:14, Dann Corbit wrote: >>> >>>>On March 23, 2004 at 18:18:51, Uri Blass wrote: >>>> >>>>>On March 23, 2004 at 17:28:17, martin fierz wrote: >>>>> >>>>>>On March 23, 2004 at 17:13:46, Aivaras Juzvikas wrote: >>>>>> >>>>>>>On March 23, 2004 at 16:40:46, Renze Steenhuisen wrote: >>>>>>> >>>>>>>>On March 23, 2004 at 16:38:28, Aivaras Juzvikas wrote: >>>>>>>> >>>>>>>>>forgot to mention, i dont try null move on 0 ply >>>>>>>> >>>>>>>>Than what's your test set? >>>>>>> >>>>>>>test set?i just let two versions of my engine play each other a couple of 15 0 >>>>>>>games, the result is either a draw or a win for the one w/o null move, even tho >>>>>>>it searches deeper as i already mentioned >>>>>> >>>>>>"a couple" meaning...? >>>>>> >>>>>>if it's two games, forget it. if it's 10 games, forget it too. start believing >>>>>>it when it's 100 games... >>>>> >>>>>I think that if you do not get improvement with null move based on 10 games then >>>>>there is good chance that you have a bug in the implementation. >>>> >>>>Null move, implemented incorrectly, can make the program play much weaker most >>>>of the time but even much better some of the time. >>>> >>>>Suppose (for instance) that R=4/6 is selected instead of 2/3 by some accident. >>> >>> >>>It means that it is not implemented correctly. >>>In the relevant case I understood that R=2 was used. >>> >>>>After ten games, it might look very good because of random chance. But 100 >>>>games would show that it was bad. >>>> >>>>I would never believe any result of less than 30 games can be trusted. >>> >>>I do not suggest to trust result of 10 games to decide if there is an >>>improvement. >>> >>>I only say that there is a good reason to believe that there is a problem in the >>>implementation after seeing bad result in 10 games. >>> >>>I do not claim that you can be sure about it but the question is what to do >>>next(play more games or look at the code to see if there is some problem in the >>>code). >> >>The point I was making is that it might look better, even though it is really >>much worse. Especially with a thing like null move, some positions would >>benefit greatly from massive pruning, but others would miss very important >>variations. >> >>So if after ten games you see 10-0, you might decide it is a great improvement >>and even tweak something else. That would be a mistake. > >I think that 10-0 is a very significant result. > >6-4 or even 7-3 may be misleading but not 10-0. With a coin toss, you can almost dismiss it. One chance in 1024 that it happens one way, and one chance in 1024 that it happens the other. So the odds are one in 512 of a complete blanking between two evenly matched opponents, in general. But with something like null move that is very, very sensitive to dangerous positions, I can easily see how a bad value will make it win a bunch of games by accident because it avoids these hairy positions by some chance. Because it will search far deeper, it will definitely play better in some "safe" sorts of positions. Personally, 10-0 means very little more to me than 6-4. I always wait for 30 games before I put any trust in it. And for 200 games before I put real faith in it.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.