Computer Chess Club Archives




Subject: Re: Proving something is better

Author: Andrew Dados

Date: 19:36:38 12/18/02

Go up one level in this thread

On December 18, 2002 at 22:12:43, Omid David Tabibi wrote:

>On December 18, 2002 at 22:08:39, Andrew Dados wrote:
>>On December 18, 2002 at 22:00:08, Omid David Tabibi wrote:
>>>On December 18, 2002 at 21:48:03, Andrew Dados wrote:
>>>>On December 18, 2002 at 18:56:21, Omid David Tabibi wrote:
>>>>>On December 18, 2002 at 18:12:22, Bruce Moreland wrote:
>>>>>>On December 18, 2002 at 16:59:10, Omid David Tabibi wrote:
>>>>>>>You take two numbers and draw a very general conclusion. Look at other tables
>>>>>>>and depths, which show a more significant superiority of std R=2 over std R=3.
>>>>>>>Look at Tables 2 and 6. Vrfd R=3 solved almost the same number of positions as
>>>>>>>std R=1 !!! Does it leave any room for doubt as for vrfd R=3's superiority over
>>>>>>>std R=3 ?
>>>>>>I don't see anything that shows demonstrated superiority of R=2 over R=3.  You
>>>>>>say to look at table 2 -- so do I.  It shows that R=2 gets one more correct
>>>>>>through ply 10, but takes over twice as long to do it.  I suggest that if R=3
>>>>>>were allowed to continue until R=2 is finished, that it would have found
>>>>>>significantly more than 1 solution in the mean time.
>>>>>>Table 6 has no node counts, so I don't know how much faster R=3 is than R=2.  It
>>>>>>gets 286 as opposed to 292.  Fine.  How much less time did it take to get it?
>>>>>>Maybe VR=3 is better than R=3.  The paper should allow me to draw this
>>>>>>A reason that I bring up the comparison between R=3 and R=2, is if you are
>>>>>>proving that R=3 is better than R=2, and you don't think that R=3 is better than
>>>>>>R=2, then maybe your other results are flawed.
>>>>>>You are writing a paper on some aspect of biological science, and your data is
>>>>>>suddenly implying that evolution doesn't take place.  Doesn't *that* seem worth
>>>>>>Either you are on the verge of a serious breakthrough, or your testing process
>>>>>>is wrong.  You need to figure out which.
>>>>>Apparently we are not looking at the data from the same perspective. As I told
>>>>>you before, I conducted self-play matches, and their results showed that std R=2
>>>>>is superior to std R=3. Although I still think that this finding is not worth
>>>>>publishing, as it is an already known fact.
>>>>>I understand your criticism of the fixed depth method, which is the standard
>>>>>scientific comparison in computer chess. But I'm afraid your case against fixed
>>>>>depth is not strong enough to convince the whole computer chess research
>>>>>community to opt for fixed time comparisons instead.
>>>>>Mentioning some fixed time experiments in a footnote or appendix could have been
>>>>>interesting; but even without them, my experiments took more than 6 months
>>>>>24h/d, 7d/w.
>>>>>If you have a specific experiment in mind, I would be glad to conduct whenever I
>>>>>get the time, but besides that, I would like the implemented algorithm in your
>>>>>program to speak for its own.
>>>>>In our discussion today, I didn't get into details and kept my replies short,
>>>>>because none of your points were new, and I have already discussed all these in
>>>>>detail a few weeks ago. I'm sure anyone who followed those discussions could
>>>>>have answered all your questions.
>>>>>Based on the programmers' feedbacks I additionally posted several implementation
>>>>>suggestions for the various variants of this algorithm, which I'm sure you'll
>>>>>find helpful.
>>>>>Now you will have to excuse me for not being able to continue the discussion,
>>>>>for I am up to my ears busy working on another paper (on Blockage Detection)
>>>>>which I hope to be ready soon.
>>>>Please, refrain...
>>>Sorry Andrew, I didn't understand what you meant.
>>I meant refrain from producing tons of 'scientific' papers with wrong or
>>unjustified conclusions. Instead of vr=3 and 80% more nodes overhead simply
>>implement checks in q-search and get more tests right with 10-15% overhead.
>>Btw.. try self-playing vr=3 versus r=2 or 3 + checks in qsearch :)
>And why do you think that vrfd R=3 won't be superior to std R=2 if they both use
>checks in q-search?

Then what do you need to verify? Biggest problem with null move is when it drops
you right into qsearch and you don't do checks after that.

Another (much smaller) problem is when a piece is trapped. Here you may try
improving on verification search: When null fails low (I know, you do
verification when null fails high) and is refuted with direct capture: either
suppress null move in subtrees, or supress it only when you try evasions of
piece en-prise. Or extend. Lots of options, most of us tried those some time
ago. This is all old stuff....

But I don't think bloating tree by 80% to get null problems right is a good

>>>BTW, are you still working on Rookie?!
>>No, I stopped like 1 year ago.

This page took 0.01 seconds to execute

Last modified: Thu, 07 Jul 11 08:48:38 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.