Author: Vincent Diepeveen
Date: 18:08:47 12/03/99
Go up one level in this thread
On December 03, 1999 at 16:42:31, Will Singleton wrote: >On December 03, 1999 at 08:32:39, Vincent Diepeveen wrote: > >>Current DIEP version, so no tricks, no extra extensions turned on >>as the ones that are currently turned on (all are out except check >>and a passed pawn extension), limited at >>6 ply against amateur at 6 ply. >> >>So limiting *all moves* it to 6 ply, including endgame. >>then DIEP at 10 ply against amateur at 10 ply. >> >>No pondering as we play with near to infinite time. >>If you want to i can also run everything on a single processor, so >>that transpositiontable luck because of parallellism >>at 10 ply is not gonna get blamed for the higher % afterwards by some >>scientific dudes that agree with you. >> >>Just searching n ply against n ply, without tricks as suggested some >>years ago like modifying eval to be an eval that did a 4 ply search. >> >>Just a 2 minute change that turns off time-check in program and forces >>it to >>use the depth=n setting. >> >>What i predict is a much closer to 50% score for the 6 against 6 ply >>match, >>and a complete annihilation of amateur when both searching at 10 ply. >> >>I assume here already of course that DIEP's eval is better than >>Amateurs. >>If i would consider the evaluation of amateur a lot better then i would >>obviously predict the opposite outcome (amateur annihilating diep at >>10 ply and nearer to 50% score at 6 ply). >> >>Of course things like learning and such must be turned off. >>Preferably both versions playing the same openings. However by just >>playing a large quantity of games we can measure statistically accurate >>what has happened. >> >>To make the experiment even better we should log the evaluations too >>and see where things went wrong and calculate whether that had >>statistical influence on the outcome. >> >>Also we could start with the same book if you want to. >> > >Vincent, > >An interesting offer, sounds like fun. However, it would be worthwhile only if > >1) Our programs were bugfree. Mine is not. >2) Extensions are identical. >3) Search and Qsearch are identical. >4) HT and null-move are identical. >5) Books identical. >6) Independent verification of all this. > >What you are trying to measure is the proposition that, given unequal evals, >games between deep searchers will enable the better eval to win more often than >games between shallow searchers, due to the vagaries of tactics (which you >contend are pretty random at 6 ply, but not at 10 ply). I contend that tactics >would continue to be significant at 10 ply, though *perhaps* not to as great a >degree as 6 ply. > >So I think it would be a worthwhile experiment, but not using unequal programs. >Use unequal evals, with the same program. i don't see it like this. i find it completely wrong to do the experiment with the same program and 2 different evals, as i don't have an eval available. Eval & search are one tuned thing to some extend. A very fast eval would call for a completely different search in DIEP, as i'm carefully selecting nodes in qsearch, not pruning on alfa etcetera. Diep is used to get outsearched pathetically (at dual 550 i again saw several 7 and 8 ply searches at 15 10 level against MrZ from Secret which is at a dual celeron 550 linux, so comparable with a dual 550 PIII NT), therefore i extend a few things, but i never extend it very deep in the current version. I can't imagine you tuned your program such that you assume for example it only makes a pawn move when it sees tactical need for it. Personally i find that crafty and ferret are a bit tuned to that. Recently a change in code: majority, seemed to cause therefore big confusion at Bob. So in short i don't see any ground why it would not be a good way to compare. What's wrong with it? Don't tell me that at 6 ply you see way more than at 10 ply! In diep i'm NOT using deep-search extensions or whatever that might confuse the test. No cheating as i said! Diep will log all searches it has done (time used, search depth, score, principal variation). this is an already existing switch in the diep.ini file. (logmoves). I asssume in amateur you are more or less doing the same? >I will make my eval suck even more than it does now, and play a 100 game match >at 6-ply and then at 9-ply levels (mine won't reach 10 too often, besides >there's some bug that causes an overflow in something when I get too deep). > >If you or others wish to do the same experiment, then we might be able to get >some usefuls results. > >Will
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.