Author: martin fierz
Date: 21:42:23 06/24/02
Go up one level in this thread
On June 24, 2002 at 10:43:34, Sven Reichard wrote: >On June 24, 2002 at 09:22:41, Vladimir Medvedev wrote: > >>Many chess programs, both commertial and amateur, are announced as "learning" >>ones. Of course, their learning ability is something different than human's one. >>AFAIK, there are two main cases of learning: >> >>1. Opening learning, when program gets some opening statistics and then avoid >>unfortunate opening lines (as Fritz does). >>2. Evaluation function learning - but not in it's main structure, only by tuning >>some parameters and weights. Using some sophisticated techniques like temporal >>difference etc. >> >>Now the questions: >> >>Does anybody know, are there any other methods of learning? >>Does learning really improve playing strength? >>Are learning abilities really useful and fruitful feature for chess program, or >>mostly marketing bells and whistles? > >Vladimir, > >I don't know what other chess programs are doing, but learning can be taken >further. >1) In the opening, a program can learn from its experience (which is more than >what I understand from your remark about Fritz). It can remember lines that were >winning in the past, both its own and its opponent's, and play them as often as >possible. If every winning line has been refuted, i.e., each move played before >in the current position leads to a game that has been lost in the past, then >novelties can be systematically tested. In this way, the program doesn't make >the same mistake twice, and it learns from the opponent's knowledge. I'm not >sure which chess program uses that approach, but it has been used in the program >Logistello (M. Buro, Toward Opening Book Learning). > >2) Of course the main structure of the evaluation function can't be changed >(unless the program modifies its own source code and recompiles itself), but if >the evaluation function is flexible enough it can discover new features and use >them. To my knowledge, the most radical approach in this direction has been done >in Checkers (Fogel et.al.). They used a neural network as evaluation function, a >fixed 6-ply search with quiescence extension, and didn't write anything in the >program except for the rules of the game, and a basic material evaluation (just >counting pieces). Then they let the program play itself for about a year, and >then let it lose in the GameZone. There it achieved a 2000 rating (comparable to >chess ratings). However, it cannot compete with top checkers programs with >hand-tuned evaluation functions. since i'm one of the guys who wrote a top checkers program with hand-tuned eval, i think i should add a few points: -> they used a topology of the neural network which somehow represents "closeness" on the board (it was a NN with 2 hidden layers, and the initial connections from the input layer to the first hidden layer were chosen so that most were 0 except for a few squares close to each other). in some sense, they probably helped the NN there. -> i replayed some of the games in the book by fogel, and actually most of the games the program wins are due to it's 6-ply (+quiescence extensions) search - the humans just blunder a piece or two. so although the program did improve by learning something about the evaluation function, it's main asset is the search; -> that program cannot only "not compete", but would just be torn to pieces by any decent checkers program. i really think the experiment was very interesting (fogel wrote a book on it, "blondie 24"), but they just wanted to present their work as a total success (they played a match against a checkers game for kids and said "blondie beats a commercial checkers program" - pure hype...), so they sort of passed by the questions which would have been really interesting. for example: if i had replaced their NN evaluation with my evaluation function (hand-tuned), and played against their program at fixed & equal search depth (their NN was *very* slow), what would the result have been? they did try playing against a material-only program, but that is obviously a very weak eval. their approach was great, because this type of evaluation function can really learn new evaluation terms and not only change weights in existing ones. but i wonder how good the final eval actually was. a german guy once wrote me asking how to interface to my checkers engines. he worked for a company which also makes custom NNs on demand, and they think they have a better idea how to do it than fogel & co. so he wanted to run matches against my program to see how good his evolved NNs were. i exchanged a couple of emails with him, and he wanted to publish his final engine for my checkerboard interface. once he got started he still wrote me how quickly the NNs were learning but then i never heard back. i guess that means that his NNs just got beaten up by my simple checkers program :-) so although the approach is very interesting, i have serious doubts that it really works. aloha martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.