Author: Peter Fendrich
Date: 02:50:29 04/13/04
Go up one level in this thread
On April 12, 2004 at 23:07:46, Christophe Theron wrote: >On April 12, 2004 at 19:51:29, Tom Likens wrote: > >>On April 12, 2004 at 19:15:01, Christophe Theron wrote: >> >>>On April 12, 2004 at 16:44:04, Tord Romstad wrote: >>> >>>>On April 12, 2004 at 14:45:28, Christophe Theron wrote: >>>> >>>>>On April 12, 2004 at 07:50:47, Tord Romstad wrote: >>>>>> >>>>>>Assume that you make a change to your engine which improves the playing strength >>>>>>by >>>>>>about 10 Elo points. How many hours of CPU time do you need before you are sure >>>>>>that >>>>>>the change was an improvement? >>>>>> >>>>> >>>>>I would say approximately one week, and I would not even be really sure it is an >>>>>improvement. We are talking about a 1.5% improvement in winning percentage here, >>>>>it's below the statistical noise of a several hundreds games match if you want >>>>>95% reliability! >>>> >>>>Thanks, Christophe! >>>> >>>>Reading this is actually a great relief to me. I wondered if you had invented >>>>some kind >>>>of magic which enabled you to find tiny improvements in much shorter time. >>>> >>>>>And unfortunately a 10 elo points improvement is becoming rare for me. Most of >>>>>the changes I try make the program weaker, and many changes do not provide any >>>>>measurable improvement! >>>> >>>>I have no difficulties believing this. My engine is still at least 200 points >>>>weaker than >>>>yours, and I have exactly the same experience. >>>> >>>>>That's why not having a strong test methodology is totally out of question if >>>>>you are serious about chess programming. >>>> >>>>Yes. It is extremely difficult to me, because I am a very impatient person. >>>>When I make >>>>a small change to my engine, I rarely have enough time to play enough games to >>>>determine >>>>whether it is an improvement, because I have a dozen new ideas I want to try >>>>before my >>>>first test matches are finished. >>> >>> >>> >>>That's the real motivation killer. I also have many ideas and when I want to try >>>them I realize I'm currently testing another idea and that the test running will >>>not be over until next week. So I have to wait for one week before I can start >>>testing, and another week to know the result. >>> >>>In two weeks from now, my interests will clearly have switched to another idea. >>> >>>That makes computer chess programming more and more boring. >>> >>> >>> >>> Christophe >> >>I was thinking about this *exact* problem on the way home from work today. >>The only solution I could come up with was to add more computers and thus >>attack the problem in parallel. I currently have three computers I can >>dedicate to running various test matches, if I could validate an idea in >>roughly two days then this problem wouldn't be so bad. A week, as both you >>and Tord point out, is difficult. Adding more CPUs to the problem would >>make this possible, but it might also turn me into a bachelor again!! >> >>As I mentioned previously, testing is my primary focus for the next few weeks. >>If I come up with anything interesting I'll share it (not being commercial >>does have a few advantages). Also don't hesitate to *not* share anything >>since you make your living at this I can appreciate your position. If >>someone asked me to design an integrated circuit for free, I might be >>reluctant to do so (especially, if it resulted in my not being able to design >>one for a paycheck in the future). >> >>regards, >>--tom > > > >I consider that I *do* contribute to computer chess programming. Not by >providing code, but by providing advices. > >I'm not doing what Bob does. Bob provides excellent advices on code, or code >structure. > >I provide more general, or philosophical, advices. They do not cover the same >areas as advices provided by other people. Some of them took me years to come up >with, so from my point of view they are valuable, maybe more than code. > >On the other hand I'm still learning myself, so sometimes my advices are not >that clever. :) > >Further, wouldn't you just *hate* if I took the fun out of chess programming by >telling you everything? :) Please tell me everything! ;-) /Peter
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.