Author: Uri Blass
Date: 13:53:08 03/24/04
Go up one level in this thread
On March 24, 2004 at 16:18:16, Dann Corbit wrote: >On March 24, 2004 at 15:30:32, Uri Blass wrote: > >>On March 24, 2004 at 14:32:01, Dann Corbit wrote: >> >>>On March 24, 2004 at 02:11:34, Uri Blass wrote: >>> >>>>On March 23, 2004 at 21:28:14, Dann Corbit wrote: >>>> >>>>>On March 23, 2004 at 18:18:51, Uri Blass wrote: >>>>> >>>>>>On March 23, 2004 at 17:28:17, martin fierz wrote: >>>>>> >>>>>>>On March 23, 2004 at 17:13:46, Aivaras Juzvikas wrote: >>>>>>> >>>>>>>>On March 23, 2004 at 16:40:46, Renze Steenhuisen wrote: >>>>>>>> >>>>>>>>>On March 23, 2004 at 16:38:28, Aivaras Juzvikas wrote: >>>>>>>>> >>>>>>>>>>forgot to mention, i dont try null move on 0 ply >>>>>>>>> >>>>>>>>>Than what's your test set? >>>>>>>> >>>>>>>>test set?i just let two versions of my engine play each other a couple of 15 0 >>>>>>>>games, the result is either a draw or a win for the one w/o null move, even tho >>>>>>>>it searches deeper as i already mentioned >>>>>>> >>>>>>>"a couple" meaning...? >>>>>>> >>>>>>>if it's two games, forget it. if it's 10 games, forget it too. start believing >>>>>>>it when it's 100 games... >>>>>> >>>>>>I think that if you do not get improvement with null move based on 10 games then >>>>>>there is good chance that you have a bug in the implementation. >>>>> >>>>>Null move, implemented incorrectly, can make the program play much weaker most >>>>>of the time but even much better some of the time. >>>>> >>>>>Suppose (for instance) that R=4/6 is selected instead of 2/3 by some accident. >>>> >>>> >>>>It means that it is not implemented correctly. >>>>In the relevant case I understood that R=2 was used. >>>> >>>>>After ten games, it might look very good because of random chance. But 100 >>>>>games would show that it was bad. >>>>> >>>>>I would never believe any result of less than 30 games can be trusted. >>>> >>>>I do not suggest to trust result of 10 games to decide if there is an >>>>improvement. >>>> >>>>I only say that there is a good reason to believe that there is a problem in the >>>>implementation after seeing bad result in 10 games. >>>> >>>>I do not claim that you can be sure about it but the question is what to do >>>>next(play more games or look at the code to see if there is some problem in the >>>>code). >>> >>>The point I was making is that it might look better, even though it is really >>>much worse. Especially with a thing like null move, some positions would >>>benefit greatly from massive pruning, but others would miss very important >>>variations. >>> >>>So if after ten games you see 10-0, you might decide it is a great improvement >>>and even tweak something else. That would be a mistake. >> >>I think that 10-0 is a very significant result. >> >>6-4 or even 7-3 may be misleading but not 10-0. > >With a coin toss, you can almost dismiss it. One chance in 1024 that it happens >one way, and one chance in 1024 that it happens the other. So the odds are one >in 512 of a complete blanking between two evenly matched opponents, in general. But there are draws so calculate 3^10/2 It can happen but very rare. > >But with something like null move that is very, very sensitive to dangerous >positions, I can easily see how a bad value will make it win a bunch of games by >accident because it avoids these hairy positions by some chance. Because it >will search far deeper, it will definitely play better in some "safe" sorts of >positions. > >Personally, 10-0 means very little more to me than 6-4. I always wait for 30 >games before I put any trust in it. And for 200 games before I put real faith >in it. The difference is more important and 10-0 is clearly more telling than 19-11 Uri
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.