Author: Frank Quisinsky
Date: 12:04:09 08/29/99
Go up one level in this thread
Hello Robert, >>example ... >>Crafty thinking for move 28 in the game >>02:58 13/02 move Ka1 without ponder >>02:20 13/04 move Ka1 with ponder >that makes no sense. pondering saved 38 seconds? It should save more like >2 minutes there. An bad example from me, but I mean that when Crafty 2 minutes more time Crafty found in 30% ponder hints not more then 5 avoidable better moves. And this 5 moves which play Crafty without ponder must not been bad ! And I will say that this is not for an statistic relevant. Bob you can see the rating list from Kai, Christian and me of the new WinBoard site. Crafty play with 2494 ELO and Comet play with 2445 ELO (over 500 games). And when I make an rating list on two PCs I think that Crafty play with ~ 2500 ELO and Comet with ~ 2450 ELO + 20-40 for ponder ! And when Comet the time control better use then Crafty play Comet with 2440 ElO and Crafty with 2500 ELO on one PC ! Or will you say that Crafty play more than 50 ELO better then Comet on one PC or better than 80 ELO by AnMon, looked in the ratinglist from Kai, Christian and me ? >>In move 29 in this game >>04:45 11/04 move Ka2 without ponder >>05:38 11/05 move Ka2 with ponder > >ditto... it depends on how long the opponent thinks _after_ crafty >starts pondering... If it thinks for the normal amount of time, crafty >gets that much think-time _free_. And I've _never_ seen the prediction >rate below 50% against a computer, more commonly it is well above 50%. >The log file will show how many moves it correctly predicted, which will >tell how many times it could potentially save time. > >But you are totally missing the point Ed raised and I seconded: if one >program has been tested and tuned for ponder=off play, and the other has >not, then that program has a significant advantage. Tough luck, you say? >Of course... but then your results don't have anything to do with how the >two programs would perform on separate machines. Yes I see that problem Robert. And I must say this is all correct what you writing ! But you think ponder make 50-100 and the time control for matches on one machine is bad (I mean, you are the programmer and you can this say) but I think ponder is 20-40 ELO and I see not time problems in Crafty when I looked this matches with longer time control. The engine which had an better time control for matches on one PC had an minmal advantage, I think 10 ELO. This advantage is not relevant. >That is why we keep saying "don't run games on one computer... the results >are not always as meaningful as you might assume..." And I say play matches on one Computer than the results are for a statistic very good. And I am happy when user play tournament with Winboard and send me this data for the homepage from volker and me :-)) >you are missing the point. my time allocation _depends_ on saving time by >pondering. You are not allowing it to do that. Which is the problem with >this... nobody would argue that _all_ engines are 50-100 elo stronger with >ponder=on than they are with ponder=off. That is easily testable on a chess >server. But the issue here is whether a program is tested with ponder=off or >not. Mine isn't. Ed's isn't. No I see this point ! And I will not say no when the programmer say yes. I will not so discussion. But Robert in this point I see not 50-100 ELO, when Crafty play with an good time control under WinBoard. And another point is all engines, yes ! OK what can an programmer make with ponder. Ponder is ponder. Programm A found the best moves in 10 seconds and play this moves in 3 minutes and programm B found the move in 3 minutes and play this move with ponder. Then had programm B an advantage ! And another advantage for ponder, learning ? And Server ... This is right, on Server the most games are blitz games. And here is ponder at the moment importent. >generally 2x faster is 70 Elo better. Pondering has the potential to make >a program act like it is twice as fast... Is this gereally 2xfaster 70 ELO better ? In the last years I think ! You say with this statement ... AMD K6-3 450 2500 ELO AMD K6-3 900 2570 ELO AMD K6-3 1800 2640 ELO AMD K6-3 3600 2710 ELO I think when Crafty on an AMD K6-3 450 play with 2500 ELO and come in Ply 13 (tournament play) the AMD K6-3 with 3600 come not in play 18 for 2700 ELO !!!! The AMD K6-3 with 3600 MHz come Crafty in Ply 15 and play with 2625 ELO ! >But suppose you take his car, and suddenly make him run with rain tires when he >hasn't in the past. How do you think he'd do then? No testing? He'd be pretty >unlikely to even finish the race. This is a common NASCAR problem in the USA. >There are many good rain tires, and some NASCAR races are on wet tracks. But >the drivers don't use the rain tires because to quote one this week "It would >be on-the-job-training, because we can't have rain when we need it to test..." > >That is the point with chess. You are testing the programs in a mode where _we_ >don't test them. Poor performance is not unexpected... Yes this is an good example :-)) OK Bob, I play with many chess programs and I have play with two computers and with one computer. My ELO is not so big than I can say it is 20-40 ELO, but I can see that the programs with ponder not play more than 5 another moves in the games. And this 5 moves which the engines play without ponder are not bad. So I will say that this is not importent for an statistic. Kind regards Frank
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.