Author: Robert Hyatt
Date: 08:04:29 07/10/02
Go up one level in this thread
On July 10, 2002 at 00:47:06, Peter Kappler wrote: >On July 09, 2002 at 19:42:37, Robert Hyatt wrote: > >>On July 09, 2002 at 17:41:09, Uri Blass wrote: >> >>>On July 09, 2002 at 15:25:17, Robert Hyatt wrote: >>> >>>>On July 09, 2002 at 13:30:55, Marc van Hal wrote: >>>> >>>>>On July 09, 2002 at 02:36:22, Uri Blass wrote: >>>>> >>>>>>On July 09, 2002 at 01:34:04, John Reynolds wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> If I understand correctly, Diep is using a Supercomputer, shouldn't it be doing >>>>>>>much better in this tournament, or is it to early to Judge? I mean the Computer >>>>>>>World Championship ofcourse. >>>>>> >>>>>>You did not understand correctly >>>>>> >>>>>>see http://www.talkchess.com/forums/1/message.html?238965 >>>>>> >>>>>>I also read that in another post that the prices for one hour of the super >>>>>>computer are very high so I guess that people need to be rich in order to use >>>>>>the super computer. >>>>>> >>>>>>I guess that in order to use the super computer you need a lot of hours of >>>>>>testing in the super computer to see that things work and if you need to pay >>>>>>some hundreds of dollars for an hour then it is something that most programmers >>>>>>cannot even consider and I talk only about 60 cpu's because the prices for 1024 >>>>>>cpu's are even higher. >>>>>> >>>>>>Uri >>>>> >>>>> >>>>>In fact I saw the statements of the WCCC and I ad once was thinking some >>>>>programs will perform worse if they are just installed on a computer >>>>>Leading to false results >>>>>All program with learning have trouble with this only one more then the other. >>>>>I don't know the reason of this but I do know this from expierince. >>>>>But in fact it is like a Tournament player who prepared his games and when he >>>>>has to play the tournament he has to forget everthing he prepared. >>>>> >>>>>Marc van Hal >>>> >>>> >>>>There are several issues: >>>> >>>>1. using unusual hardware is non-trivial. NUMA machines are one example. >>>> >>>>2. Going faster may well cause your eval to misbehave as it is very easy to >>>>tune an evaluation to a specific search depth and going much deeper or shallower >>>>can cause some of that tuning to be wrong. >>> >>>I agree about the other problems but 2 is not a serious problem. >> >> >>First question, have you _ever_ done this? I have. And I have been burned >>by it. >> >>Second question, did you ever see my comments about how we almost lost (or >>didn't win) the 1986 WCCC event due to this _very_ problem? If not, I can >>re-tell the story again. >> > >Yes, please re-tell. Here is what happened. During the early Summer, Bert Gower and I were playing nightly games using Cray Blitz running on a vax vs several different micros. We were running about 100 nodes per second and doing 5 ply searches. We were doing this to get ready for the 1985 ACM computer chess tournament in Denver, and we knew I was moving to Birmingham in August to start work on my Ph.D. We noticed that we were generally tactically stronger than any of the micros we were playing against, and in tactical situations, we won easily. But on occasion, we would create a "hole" in our pawn structure that would hurt us in the endgame where the opponent could infiltrate off-side and cause trouble since we were both doing pretty shallow searches. To fix these holes, I added two lines of code into the pawn evaluation loop in Cray Blitz and further testing showed that we had "closed the door" on the pawn hole problem and we lost no more games due to that problem. When we got to denver we played not very well and lost 2 of 4 games, with the new HiTech machine winning the tournament. I was neck-deep in my first year of Ph.D. classes and just thought "seems that several have 'caught up' with us" and let it go at that. The next year, at the WCCC tournament in Cologne, we won easily in the first round, but lost in the second to a program that really had no business beating us (we were running on an 8 cpu Cray I believe). After losing in round 2, we started looking to see what was wrong. It was playing _very_ passively with its pawns and its score would climb as the opponent pushed his pawns. On a "whim" I commented out the two lines of code dealing with pawn holes. Suddenly it started finding the moves it used to find. In games where it played horribly passively, it suddenly started to play aggressively as it used to. We left those two lines of code out, and we easily won the next three rounds, including beating HiTech in the final round in a convincing way. A way so convincing that Berliner accused of of cheating because "no computer would play some of those moves." A change that worked _great_ at 100 nodes per second and five ply searches, turned out to be terrible at 9-10 plies. After a lot of study, it turned out that the pawn hole code was covering a hole that other eval terms were handling with the deeper searches. Which basically made holes twice as bad at 10 plies as they were at 5 plies. Removing two lines completely changed the character of the program, and it also _clearly_ shows that an eval for a shallow search can be quite different than what is needed for a significantly deeper search. > > >>Believe me it _is_ a problem. From someone who developed a chess engine on >>a machine running 100 nodes per second, and then played on a machine >>searching 1000 times faster. It can be a _serious_ problem. >> >> >>>Every program that I know is going to play better if you give it 10 hours per >>>move and not 3 minutes per move. >>> >>>Uri >> >>Sorry, but you don't know "every program". >> > >Nor did he claim to. > >I can't see this happening (weaker play with 200x speedup) unless you have a >major bug like a sign error in a large positional term like passed-pawn scoring >or king safety. But that's far beyond what I would consider a "badly-tuned" >eval. What if you only do very shallow searches. So shallow that you can't under- stand some simple positional idea. You add eval code to handle that. Then you search far deeper on different hardware and you both see the consequence of the positional idea thru search, _and_ you see it thru eval. A "double dose". If you tune the positional term down so that it doesn't double-deal you on fast hardware, it is ineffective on slow hardware. If you turn it up so that you don't make the positional mistake on slow hardware, you might find yourself paranoid about the problem on fast hardware since you see twice the penalty. That is what happened to us. Could it be avoided? No idea. But it _clearly_ can happen. It _did_ happen for one well-known program... Mine... > >Actually, forget that, I think the entire eval would have to be backwards for >that much extra speed to weaken you. I'd gladly invert just my king safety for >a 200x speedup. ;) > >-Peter If you play on ICC long enough, you learn that you have to "turn up your king safety" to survive against IM/GM players in bullet/blitz games. Otherwise they attack you before you know you are being attacked. You then discover that this makes you very passive in long time controls. And you have to turn it back down or else you play so passively you get squeezed to death. Happens to most everyone that plays there. You get the right "setting" for say 1 0 bullet, then at game/60 minutes you become way too passive. Or vice-versa...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.