Author: Robert Hyatt
Date: 19:39:16 10/09/02
Go up one level in this thread
On October 09, 2002 at 18:47:53, martin fierz wrote: >On October 09, 2002 at 18:28:01, Robert Hyatt wrote: > >>On October 09, 2002 at 18:25:05, Robert Hyatt wrote: >> >>>On October 09, 2002 at 16:46:09, martin fierz wrote: >>> >>>>On October 09, 2002 at 16:20:52, Robert Hyatt wrote: >>>> >>>>>On October 09, 2002 at 14:44:45, martin fierz wrote: >>>>> >>>>>>On October 09, 2002 at 14:23:03, Robert Hyatt wrote: >>>>>> >>>>>>>On October 09, 2002 at 13:28:50, Mike S. wrote: >>>>>>> >>>>>>>>Percentages, based on a large comp-comp database: >>>>>>>> >>>>>>>>Engine | #Games total W B | total eQE* W/eQE B/eQE >>>>>>>>------------------------------------------------------------------- >>>>>>>>Fritz 7 | 784 69% 72% 65% | 59% (#57) 53% 67% >>>>>>>>Chess Tiger 14 | 850 66% 71% 62% | 72% (#71) 73% 71% >>>>>>>>Shredder 6/-P. | 743 61% 65% 57% | 58% (#58) 63% 53% >>>>>>>>Junior 7 | 799 55% 58% 53% | 41% (#60) 25% ! 56% >>>>>>>> >>>>>>>>*) "eQE" = early queen exchange (within the first 10 moves) >>>>>>>> >>>>>>>>Fritz 7's white percentage after an early exchange of the queens was 53% only, >>>>>>>>compared to it's general white average of 72%! Remarkable also Tiger 14's result >>>>>>>>with black: Much better (71% to 62%) without queens. Desastrous was Junior 7's >>>>>>>>result with white when the queens were off the board soon: only 25% (in 30 games >>>>>>>>of that kind). >>>>>>>> >>>>>>>>It looks as if the engines each are very different, in how they depend on having >>>>>>>>the queen... with Shredder 6/-Paderborn showing the smallest impact. >>>>>>>> >>>>>>>>For games with Black against Fritz 7 or Junior 7 (and probably against others >>>>>>>>too for which I didn't search the statistics), it could be promising to have an >>>>>>>>opening book which favours eQE variants... But that of course must not have >>>>>>>>"wholes" in other (more common) lines, so it can't be done by simply generate an >>>>>>>>opening tree based on an eQE games database only. >>>>>>>> >>>>>>>>Regards, >>>>>>>>M.Scheidl >>>>>>> >>>>>>> >>>>>>>I don't think it is too surprising. It just highlights a weakness that programs >>>>>>>fail to understand >>>>>>>basic endgame ideas, and rely more on tactics than on knowledge to move along >>>>>>>thru a game. >>>>>>> >>>>>>>If a program has some basic holes in its knowledge about endgames, then removing >>>>>>>the queens >>>>>>>is going to highlight those holes. Or, the inverse, keeping queens on tends to >>>>>>>cover up those holes, >>>>>>>at least for a while. >>>>>>> >>>>>>>Programs that don't understand majorities, weak pawns, distant >>>>>>>majorities/passers, which minor >>>>>>>pieces work best with pawns in various configurations, the fact that pawns on >>>>>>>both wings give >>>>>>>better winning chances than pawns on one wing, etc, are going to have great >>>>>>>trouble with GM >>>>>>>players. >>>>>>> >>>>>>>I've given some examples of things I've had to fix after watching GMs pick on >>>>>>>the same hole >>>>>>>over and over. Today I don't see those huge holes cause me a lot of trouble >>>>>>>(yes I still have >>>>>>>holes, to be sure, but not the building-sized holes some "tactical" programs >>>>>>>possess..) and I >>>>>>>don't particularly care if queens come off early or not. If you hear someone >>>>>>>complain about >>>>>>>an early queen trade, you can rest assured they _know_ they have some serious >>>>>>>endgame >>>>>>>holes that need work... >>>>>>> >>>>>>>And you can also rest assured that after the kind of practice Kramnik has had >>>>>>>with Fritz, that >>>>>>>he _knows_ what kind of holes are there and he's going to park in them every >>>>>>>day, since they >>>>>>>can't be fixed due to match rules (stupid rules I might add). >>>>>> >>>>>>just wondering: what do you think fritz' problem is and how would you fix it? >>>>>>and do you think you could fix these problems in a single day? >>>>>> >>>>>>aloha >>>>>> martin >>>>> >>>>> >>>>>I don't think many problems are "single-day" deals nowadays. At least not the >>>>>ones I >>>>>have seen and/or fixed in my code. The first problem is figuring out what the >>>>>"hole" is. >>>>>The last major one I had was pointed out by a GM (Roman). He was patiently >>>>>waiting >>>>>for me to log on one day and could not wait to "get started". >>>>> >>>>>"Bob, you _must_ do something about this endgame hole. cptnbluebear has won >>>>>four >>>>>games in a row with the same theme." >>>>> >>>>>"what is it?" >>>>> >>>>>"Crafty thinks two connected passed pawns are much stronger than two isolated >>>>>passed pawns >>>>>in a king and pawn ending. But the king can stop the two connected passers >>>>>easily while a >>>>>lone king has great trouble with isolated passers, the farther apart they are >>>>>the harder they are >>>>>to deal with." >>>>> >>>>>That kind of thing. Not really hard to fix. Not really easy either. But the >>>>>hard part is finding >>>>>what is wrong (recognizing it) rather than fixing it. That is a place where a >>>>>GM "helper" can >>>>>make a _huge_ difference. >>>>> >>>>>Fritz (and most commercial programs) are tuned in coarse ways. Big passed pawn >>>>>bonuses. >>>>>Big king safety bonuses. Take away those two features, and it has trouble, >>>>>because it doesn't >>>>>seem to understand that a majority will turn into a passer one day, but since >>>>>the search can't see >>>>>the passer it doesn't consider it at all. Controlling open files is fine. _if_ >>>>>the file is useful and >>>>>_if_ you _really_ control it. I've seen more than one program park a rook on >>>>>(say) the open >>>>>E file, even though black can block that file at will with a bishop. The rook >>>>>can't go to the >>>>>7th because Be6 locks it in. But it is as happy as a pig in slop with that file >>>>>even though it can't use >>>>>the squares on the opponent's half of the board in any way. >>>>> >>>>>General knowledge. With specific failures. Kramnik is exploiting this >>>>>ruthlessly... >>>> >>>>yes, i can see that. but my question was directed at your "stupid rules". if the >>>>problems in fritz are not simple fixes, as you seem to admit, then what would >>>>the fritz team gain by being allowed to toy around with the program? i don't >>>>know if i would dare to change anything in a well-tested engine in the middle of >>>>a match without time for testing :-) >>> >>> >>>There are things to tweak anyway. IE "be more aggressive" so that it will play >>>differently in the second pair of games than in the first, assuming you lose the >>>first pair. Or "be less aggressive". Or other such things. The main thing is >>>to >>>_not_ become too predictable, as that is the kiss of death against a strong GM. >>> >>> >>> >>>> >>>>also, when you write "fritz & commercials are tuned in a coarse way", does this >>>>imply you think crafty or any other program is tuned better? >> >> >>Missed that one. No. What I mean is that the commercial programs are _really_ >>tuned to >>play against other commercial programs. Often that simply requires a new piece >>of knowledge >>with a relatively big score. IE Fritz seems to over-evaluate weak pawns. >>Kramnik showed it >>that sometimes three isolated pawns are _not_ a weakness at all. Yet against >>computers, the >>idea Fritz does is probably perfectly acceptable. >> >>Too bad you couldn't listen in on the many GM conversations I have had, where >>they pick up on >>a specific thing (bishop pair too valuable, bishop pair not valuable enough, >>connected passers >>too valuable, etc...) Because once they see that, look out... > >i haven't had the experience as a programmer, but i've played a bunch of GMs. >what actually surprises me most is not these general statements you make here, >but that they seem to know the exceptions to the general rules much better than >players of my strength (2200 fide) do. they will allow a doubling of pawns under >circumstances where i would think it's a weakness and then show me it's not. or >they will play a move (like g4 with Kg1) which i would discard as a weakening of >the king when they have the feel for whether the king will be in danger or not. >or they will sac an exchange for long-term chances where i can see that you >could do it but would not have had the nerve. >my feeling is that they know so many different nuances of positional play that >an eval function would have to be really HUGE to even get close to their >understanding of the game :-) > >> >>And Kramnik has had a _long_ time to see it, and the program can't be changed to >>make it >>behave differently... > >again, what do you think they could change to escape kramniks torture? he doesnt >seem to go for any specific weakness (like the ones you listed), but just for >endgames where everything the computer does seems to be wrong. we have seen: >- fritz not understanding that after h4 the game is a safe draw in game 1 due to >a blocked position >- fritz not understanding that a passive defence in a rook ending is usually >worse than an active defence giving up a pawn in game 2 >- fritz play 18.a3? giving up the bishop pair and weakening his pawn structure >in game 3. > >i don't see how much of this can be fixed with a simple change. implement a >blocked position detection in a day? mmmh - i don't think so. change your >weights in the rook ending eval without testing them so that the program might >panic in defensible positions and jettison a pawn without reason - not a good >idea. increase the value of the bishop pair overall for game 3? perhaps. add >knowledge that a2-b3-c4 is better than a3-b3-c4 in general? i'm not sure that is >a useful rule :-) > >you could change some eval weights and hope that kramnik's preparation is too >deep-fritz-7-specific. but i really don't see any evidence for that... the main >thing to me seems to be that they have to change the book, and they are allowed >to do that. > >aloha > martin The kind of changes I am thinking of simply change the program's "personality". IE you play Crafty a _bunch_ of games and discover something. Say the isolated pawn evaluation is too optimistic. It takes several games to discover that. Since the match is only 8 games long, I could make a change, and try to cover up that problem, and now you might be hard- pressed to find another weakness _quickly_. Sure, it would have one, but you would start the next game trying to play on the old isolated pawn glitch, and it would fail. And the game would be pretty "equal" at that point... The problem is that finding holes is not hard, but takes several games. Covering such a weakness up is also not hard, at the risk of creating a new and bigger weakness of course. But you won't have many games to find and exploit the new hole while probing around trying to find the old one that is now gone (or mostly gone or hidden). I agree about the "exceptions". GMs are quite remarkable in that. A rook on the 7th is great, except for (a) (b) (c) ... (exception number 500 or more).. :)
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.