Author: Otello Gnaramori
Date: 01:56:52 04/06/02
Go up one level in this thread
Question: Do programs become more human when they have more chess knowledge? CT: Programs become more human when they become stronger. Increasing search speed does not lead to less understandable chess style. The speed (or depth) of search helps the program to synthesize chess knowledge it does not have “built-in”. For example my program knows nothing about the concept of “tempo”. But if you look at its games, you will see that sometimes it makes great efforts to save a tempo. The knowledge is not there explicitely, but the program plays exactly as if it had the knowledge, because existing knowledge and search together help the program to synthesize the missing knowledge. So to answer your question it is possible to make a program more “human” either by making it search more efficiently OR by adding more knowledge in the way it evaluates positions (or both, of course!). The bottom line of this is that there is also chess knowledge in search. It is a common mistake to believe that chess knowledge is only in the evaluation functions, and that search is a mechanical, inhuman part of a chess program. It is not true. I am using a lot of chess “knowledge” to teach my program how to search. This knowledge is not the conventional one that is used by human chess players, but it is CHESS knowledge because it could not be applied to other games (it is specific to chess). It is knowledge about which variants are worth searching, and which ones can be pruned away early. Actually many human chess players are using the same kind of knowledge, but they are doing it unconsciously. It is possible for a human chess player to explain why he prefers this position over this one, but it is much more difficult for him to explain how he has been able to find a deep variant and why he believes it is going to be the normal continuation of the game. Creating better search algorithms is also a way to create more human chess programs. At this time my program must search several millions of positions, when a human GM would only search several hundreds of positions to play a move of the same quality in a given position. If I can manage to have my program searching less positions in order to play at the same strength, then obviously I'm getting closer to “humanity”. Question: Do you think that some programs need faster hardware, for example they will beat program x on 1000 MHz computers but they will lose the same program on 500 MHz? Is this so for your Tiger programs? CT: I think that if a program needs faster hardware, then it is poorly designed. My deep belief is that it is possible to write a program that will perform equally well at any time control on any computer (or almost), and that's what I am trying to do with the Tiger engines. I don't see any reason why a program could be strong only at very slow time controls or on very fast hardware. On the other hand I see this excuse very often used for programs that do not perform as well as expected: “but you played games in one hours, and this program did not perform well because it is clear that it needs 40 moves in 2 hours time controls to perform at its best”. Believe me: each time you see something like that, it is a poor excuse. If you do the test the guy will come back and say that you needed to play on Athlon 1.3GHz instead of PIII-700 MHz and so on. I remember that several years ago some people were claiming that MChess and CSTal would kill everybody on faster hardware. This was of course bullshit and has been proven wrong Question: What about your examples, like finding out about tempos only through the search? I wish you would explain “knowledge guided”. What kind of knowledge, obviously not GM or expert chess knowledge, as you already explained. Technical knowledge in the search? Algorithmic tricks? CT: Chess knowledge! Search extensions are an example. When you see an interesting or forced move, then obviously you need to see a little deeper what is going to happen. That is why most programs will look a little bit deeper at what happens after a check or a forced recapture. That is chess knowledge used to guide the search. On the other hand there are moves that are obviously stupid, and which obviously do not deserve to be searched any further. Question: What about search depth. Will in your opinion as an expert (if you can't guess who can?) going one ply deeper make less and less difference in the future? CT: Going one ply deeper will make less difference in the future. However it is still going to make a hell of difference for the coming ten years (after that, we will see). Question: How do you check the quality of your programs. Do you use test positions or do play autoplayer games? On which playing levels do you test your programs. CT: The way I test is what I would call a trade secret. Testing a chess program is as important as writing it or adding new knowledge into it. If you buy a Tiger product, you might believe that what you get, a 300 KB DLL or executable file, is the result of all my work. Actually it is only a small part of my work. Chess Tiger is a big project and the engine in itself is only the most visible part. You do not see that Chess Tiger is actually a collection of several other programs. The whole collection includes programs to test the engines, programs to debug the engines and programs to tune the engines. In particular the programs that are used to test the engines are very important and took me months to develop. These programs are fully automatic and run on several computers in my office. The test programs can run hours or even days and eventually produce a profile which tells me how good the new engine performs. My decision to keep an improvement or discard it is mainly based on the result of these tests. I think that the testing methodology is what makes the difference today between amateur and professional programs, and even between professional programs. It took me years to understand that my judgment was not enough and that I needed more objective ways of making my decisions. Then it took me years (at least 3) to build a good testing methodology. And since I'm not perfect, I question this methodology on a regular basis. It has even been the subject of many discussions with my partner Ed, who is still not in full agreement with me on this topic. Question: Is Tiger code getting bigger and bigger as you add more knowledge or can you “tune” existing algorithms to include special cases (example: pawn structures) most of the time? CT: I try to add code which is as general as possible, so Chess Tiger includes very few special cases. So the code does not have to get bigger and bigger too much. However in the endgame this is not possible. The endgame is full of special cases, that's a real nightmare to make all these bits of code working together. Question: What is your impression of the cooperation between chess-programmers: are most of them willing to help each other, is it a community of colleagues or even friends, fighting together for a common goal, exchanging ideas – or is there only a competition between them, each one trying to make the best program beating all the other programs? CT: Generally they do not hate each other, but they will not say anything to their colleagues. It is too easy in a few word to explain a successful idea, and idea you took one full year to find and get to work. And when you are a professional programmer, this idea can make all the difference. However, from what I have seen, these guys are most of the time friends with each other. I remember especially the last evening of the WCCC in Paderborn 1999. At my table there was Ed Schröder, Frans Morsch, Marc-François Baudot, Johan de Köning... This evening was well worth traveling from Guadeloupe to France, then from France to Holland, then from Holland to Germany! Question (from a chess programmer): Last November I started programming a chess engine, and I progressed from a 1500 level to 2300 Elo very quickly, just by implementing the well known algorithms and working on the evaluation function. But now things start to get very, very difficult. Any tips on how to proceed? CT: I have a very simple advice for you, but it is a valuable one. It took me some time to figure it out, so maybe you can save this time. From now on, your judgment alone is not enough to decide when a change is really an improvement. If you rely only on your feelings, you are going to turn in circles. You will add something or change something, and you might eventually realize that this change has weakened your program. For you the time of the easy improvements is gone. Every additional Elo point from now on is going to be hard to earn. What you need now is a serious testing methodology. You need to define an accurate way to decide if a change has improved the playing strength of your program or not. Playing a few games manually will not do it. You probably need to write modules in your program dedicated to testing. For example a module that will eat a set of EPD position and try to solve them, and after the engine has crunched the positions you need to be able to output some kind of statistics, and you must be able to compare the output with the output of your current reference version. Maybe you need to study a little bit of statistics (if you do not have the required background already) to understand about things like margin of errors in random events. Maybe you need to implement Auto232 in your program, so you can get a large number of automatic results. But from now on you will not make any significant progress without a serious testing methodology. So I would advice you to invest a lot of time in finding and refining yours. A lot of time means several days, probably several weeks. By the way, when I was an amateur chess programmer I would have really liked to be able to talk to some well-known programmer. Promise me, when you are at the top, spend a little bit of your time everyday speaking to the people who like computer chess. I'm sure you will find in it a motivation to keep on working an improving your engine because they are going to send you back some good energy. The complete interview is at http://www.computerschach.de/sprechstunde/archiv/theron1e.htm (in english!) Regards, Otello
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.