Author: Aaron Tay
Date: 05:28:05 05/22/03
Go up one level in this thread
On May 22, 2003 at 03:55:59, Uri Blass wrote: >On May 22, 2003 at 00:53:37, Stephen A. Boak wrote: > >>On May 21, 2003 at 08:50:15, Uri Blass wrote: >>> >>>That tells us that testers were simply lazy at that time and did not try all >>>combination of changing one parameter. >>> >>>Ed found that reducing the chess knowledge of Rebel from 100 to 25 helps but no >>>tester was able to find it. >>> >>>Uri >> >>Let's think. >> >>How many combinations of single parameters & values are there? >>How many single parameters can be adjusted on CM9000? >>How many possible numerical settings on each parameter can be used? >>Multiply the figures. Call this result 'A'. > >I do not know and I guess 300 possibilities(I do not think that changing >parameter to all the possible options is a good idea and if you see that >increasing parameter from 100 to 150 is counter productive then you do not need >to test changing it from 100 to 175). > >> >>How many games will it take, at each single parameter & value combination, to >>determine if that setting is an improvement? >> >>What will the test be? Test Suite, Actual Game play? > >I think that test suite can be a good indication if the tester was not lazy to >spend years to build the correct test suite that determines if a change is an >improvement. > >I have an idea how to do it but I was too lazy to spend some years of >computer time to build the test suite. >I also cannot be sure that my idea is right because I did not test it. > > >> >>How long does it take to 'be sure' if using Actual Game play? > >I do not know. >maybe 1000 hours but in many cases it is possible to decide after 100 hours >that the change is bad. >> >>If using Test Suite, how many minutes or hours for running the Suite one time. >>Call this answer 'B'. > >I think that in case of test suite 100 hours may be enough when in many case 10 >hours in solving part of thetest suite are enough to decide that there is no >improvement. >> >>Multiply 'A' times 'B'. What is the total minutes or hours? > >something near 100,000 hours(if games are used) or 10,000 hours(if test suites >are used) . > >200 testers when everyone of them use 500 hours and cooperate may be enough >even in case of games. >> >>Now let's think again. Is it laziness, or a lack of time (Game Play or Test >>Suites) or something else? > >lack of cooperation. >> >>It is easier say 'lazy' than to be not lazy. >> >>Regards, >>--Steve > >I agree that it is easier to say lazy than to be not lazy. >I did not criticize a specific tester in my post. > >I only think that testers could get better result in the same time of testing. They could, I dont think there's a logical proof anyway saying they couldnt.. Evidence for the case of Rebel indidcates that's it's not easy. >Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.