Author: Uri Blass
Date: 00:55:59 05/22/03
Go up one level in this thread
On May 22, 2003 at 00:53:37, Stephen A. Boak wrote: >On May 21, 2003 at 08:50:15, Uri Blass wrote: >> >>That tells us that testers were simply lazy at that time and did not try all >>combination of changing one parameter. >> >>Ed found that reducing the chess knowledge of Rebel from 100 to 25 helps but no >>tester was able to find it. >> >>Uri > >Let's think. > >How many combinations of single parameters & values are there? >How many single parameters can be adjusted on CM9000? >How many possible numerical settings on each parameter can be used? >Multiply the figures. Call this result 'A'. I do not know and I guess 300 possibilities(I do not think that changing parameter to all the possible options is a good idea and if you see that increasing parameter from 100 to 150 is counter productive then you do not need to test changing it from 100 to 175). > >How many games will it take, at each single parameter & value combination, to >determine if that setting is an improvement? > >What will the test be? Test Suite, Actual Game play? I think that test suite can be a good indication if the tester was not lazy to spend years to build the correct test suite that determines if a change is an improvement. I have an idea how to do it but I was too lazy to spend some years of computer time to build the test suite. I also cannot be sure that my idea is right because I did not test it. > >How long does it take to 'be sure' if using Actual Game play? I do not know. maybe 1000 hours but in many cases it is possible to decide after 100 hours that the change is bad. > >If using Test Suite, how many minutes or hours for running the Suite one time. >Call this answer 'B'. I think that in case of test suite 100 hours may be enough when in many case 10 hours in solving part of thetest suite are enough to decide that there is no improvement. > >Multiply 'A' times 'B'. What is the total minutes or hours? something near 100,000 hours(if games are used) or 10,000 hours(if test suites are used) . 200 testers when everyone of them use 500 hours and cooperate may be enough even in case of games. > >Now let's think again. Is it laziness, or a lack of time (Game Play or Test >Suites) or something else? lack of cooperation. > >It is easier say 'lazy' than to be not lazy. > >Regards, >--Steve I agree that it is easier to say lazy than to be not lazy. I did not criticize a specific tester in my post. I only think that testers could get better result in the same time of testing. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.