Author: Peter Kappler
Date: 17:26:59 10/05/99
Go up one level in this thread
On October 05, 1999 at 16:16:00, Robert Hyatt wrote: >On October 05, 1999 at 13:51:32, Christophe Theron wrote: > >>On October 05, 1999 at 12:10:31, Robert Hyatt wrote: >> >>>On October 05, 1999 at 11:28:10, Christophe Theron wrote: >>> >>>>On October 05, 1999 at 10:59:56, blass uri wrote: >>>> >>>>>On October 05, 1999 at 09:25:43, Wayne Lowrance wrote: >>>>> >>>>>>On October 05, 1999 at 08:42:36, Bernhard Bauer wrote: >>>>>> >>>>>>>On October 05, 1999 at 07:38:57, Shep wrote: >>>>>>> >>>>>>>>On October 05, 1999 at 07:26:32, Steve McRiley wrote: >>>>>>>> >>>>>>>>>On October 05, 1999 at 04:35:21, Didzis Cirulis wrote: >>>>>>>>> >>>>>>>>>>On October 05, 1999 at 02:45:16, Howard Exner wrote: >>>>>>>>>> >>>>>>>>>>>What is the hardware used and the time control? Thanks. >>>>>>>>>> >>>>>>>>>>As usually ;-) 200MMX, 16MB for each program, PB off, 1hour/game (means one hour >>>>>>>>>>for each program), careful calculations of CPU usage. >>>>>>>>>> >>>>>>>>>>Didzis Cirulis >>>>>>>>> >>>>>>>>>You mean you do this on one computer?? >>>>>>>> >>>>>>>>Hmm, do we really have to start this all over again? ;-( >>>>>>>> >>>>>>>>Actually, it is not necessarily bad to play on a single machine. >>>>>>>>For one, programs like Fritz and Tiger are very well suited for this as they >>>>>>>>don't hog CPU time when sitting idle. >>>>>>>>Secondly, there is no relevant evidence yet that the time management problems >>>>>>>>caused by "PB off" actually have a significant impact on the outcome - >>>>>>>>at least not more than the good old statistical problems of playing a series of >>>>>>>>less than 1000 games... ;-) >>>>>>>> >>>>>>>>--- >>>>>>>>Shep >>>>>>> >>>>>>>As Fritz and Tiger are designed to be used with PB=on any results with BP=off >>>>>>>are questionable since they do not reflect the true program strengt. >>>>>>> >>>>>>>Kind regards >>>>>>>Bernhard >>>>>> >>>>>>I agree with that. I just cannot get interested in matches played on one comp. >>>>>>If the author of both programs would come out and say it makes no difference >>>>>>that would be another story. However I know Bob Hyatt has said so many times it >>>>>>affects Crafty ! >>>>>>Wayne >>>>> >>>>>I do not agree. >>>>> >>>>>I think the difference between the result with playing in one comp and >>>>>playing in 2 computers is minor(The difference in rating is probably not more >>>>>than 20 elo). >>>>> >>>>>Uri >>>> >>>> >>>>You are completely right, Uri. >>>> >>>> >>>> Christophe >>> >>> >>>He may be right for your program. He _definitely_ isn't right for mine. It >>>does screw up time allocation. Because I _plan_ on making a few moves in >>>zero time due to pondering, and use this time early when it is needed. With >>>no pondering, I won't get it. Better (in my case) to play with pondering on >>>on both engines. This won't ever hurt crafty, but will possibly influence the >>>outcome of games. >> >>We can think of the "Permanent Brain" as a time bonus for both programs. I mean >>if you guess the opponent's move 50% of the time, you will benefit from a >>(roughly) doubled thinking time in 50% of the moves you have to play. >> >>So if your average time per move without permanent brain is T, with PB on it is >>1.5*T >> >>This 50% figure is maybe not true, but whatever the exact percentage is, it is >>anyway the same for both programs. >> >>In case PB is on, take the time you have to the next control and multiply it by >>1.5, and you should be in the right area. If PB is off, just use the time you >>have been given and don't expect to get more. >> >>If you allocate more time in the early middlegame because Crafty has problems >>handling complex situations in that stage, you could as well do it when PB is >>off... >> >>Without needing a lot of test, you can apply this idea to fix your time >>allocation algorithm when PB is off. And you will make your users happy because >>they will be able to test Crafty against anything on one computer. >> >> >> >>> Ie in endings where crafty has tablebases, it can use less >>>than 100% of the CPU when probing. That gives the opponent a 'faster' machine >>>at that point. >> >>What do you mean exactly? >> > >If you test with PB _ON_ in both programs on one computer, it is possible that >the TB probes crafty does will slow it down while it blocks for I/O. And the >_other_ program can consume those cpu cycles that Crafty isn't using. On two >computers this could never happen, but on one it can, if PB is on. If PB is >off, then timing changes cause difficulties. > > >>1) That you don't use 100% CPU time when probing tablebases? In this case it's >>up to you to do whatever you can to use 100% of your CPU... > >Care to tell me how to do a read and then not wait on the result? :) I >_must_ know the result of a TB probe before I go on and start any work at >this node... > > >>2) That your opponent is going to use some CPU resource while you are accessing >>the endgame databases? I don't think so. Either the other program runs in a DOS >>box that is STOPPED when it is in the background, or the other program is >>guaranteed to be idle when not thinking. So you can access as many files as you >>want or even sleep for a while, but your opponent is not going to do any >>computation in your back. >> > >if PB if off, the above doesn't happen, of course, _UNLESS_ you are playing >one of the chessmaster programs that seem to like to burn the cpu when they >are sitting idle. > >Testing on _one_ machine is a bad practice, period. There are too many >variables. way too many. > > > >> >> >>>Also, if the opponent has no move to ponder (due to a fail high) >>>then Crafty will get 100% of the machine. And Crafty will _never_ not have a >>>move to ponder because if it doesn't it will first search for one (puzzle >>>search) and then ponder that. >> >>And the 5 elo points you get from this smart algorithm justifies that Crafty >>should not be used without PB? >> >> > >I would think that "If I say so, yes..." I wrote the thing. I know how it >behaves. I know what assumptions it makes about the computing environment. >The next thing will be someone with two cpus running two SMP-ready programs >at the same time. This will _kill_ my program, as it was _not_ designed for >shared computing resources when doing the SMP search. Should I plan for that >too? I don't think so... > > > > >> >>>So either way, the results from 1 machine won't necessarily match the results >>>on two machines. >>> >>>It ought to be avoided, regardless of popular opinion. There _are_ problems >>>in doing this... the above are just a few of them... >> >>So to be clear you suggest that Crafty should be left out of tournaments played >>on one computer only? >> >> > >yes... > Bob, This is really a shame, because Crafty is the best known of all the winboard engines, and is often used as *the* standard for assessing the strength of other programs. Anytime I hear that "program X" is a really strong amateur engine, my first thought is "well, lets just see how it stacks up against Crafty". As much as possible, I think you should try to address some of these single-computer winboard issues - especially the no-ponder time management problem, which just doesn't seem that difficult. Problems that can't be solved should at least be quantified, performance-wise, and this information could be published. As Christopher said in his previous post - if the net effect is 5 or 10 ELO points, who really cares? On the other hand, if you can prove that for Crafty the difference is 50 or 75 ELO points, then at least people will take this into account when interpreting results. --Peter
This page took 0.03 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.