Author: Jeroen Noomen
Date: 07:42:39 04/13/98
A few days ago I have been asked by CCC (like most of you) to enter the discussion about Fritz 5. Since a few weeks I have Fritz 5 on my PC, too short a period to come to any definite conclusion. IMO there is no doubt about it that Fritz 5 is a great program: a lot of nice features, great graphics and a very strong chessprogram. One must congra- tulate ChessBase on doing a fine job here. Seeing the published games and playing games myself (Jeroen vs. Fritz 5, autoplayergames vs. other engines) I am sure Fritz 5 is one of the top programs at the moment. It has a pure tactical-based style, which is difficult to handle for human players. Out of any position Fritz seems to be able to confuse matters and to come up with tactical resources. Although I am of the opinion that 'knowledge-based' chessprograms have the future, there is no question about it that Fritz 5 plays a very active way of chess, trying to put its pieces in the right places and searching for tactical possibilities. I am disagreeing on any comment that says that Fritz 5 plays bad chess, positionally weak and so on. With any program it is possible to come up with positions in which this particular program plays weak moves. ALL programs suffer from the fact that they still are no grandmasters in positional play. To come at this point: IMO it is still far, far away before any chessprogram will reach the level of understanding the game like f.e. Kasparov, Kramnik or Anand. I read a lot of comments about the Fritz - SSDF subject. A few of them are: 1. Fritz 5 plays only with 64 MByte of Hash 2. Fritz 5 is allowed to play with endgame databases 3. Fritz 5 is using a special powerbook 4. Fritz 5 is tuned on the best programs in the SSDF-list 5. Fritz 5 cannot play with AUTO232 I don't know why these subjects are brought up, there seems to be nothing wrong with this. IMO this is all very normal and there is nothing 'illegal' in it. F.e. take the point of tuning: Are not all programmers doing this? It's absolutely nonsense to attack ChessBase for something that others are doing for years! There ar however a few points that should be cleared: If a neutral organisation like SSDF are testing chessprograms, the conditions for these tests should be as equal as possible. That means: If you present results based on Pentium 200 MMX, these should be the SAME Pentiums, with the same configuration. Another point: the ChessBase autoplayer. One can ask the following questions: 1. Why does a commercial organisation make an investment in a project (like f.e. an autoplayer) if something similar already exists? 2. Why comsume a lot of time and money to build a piece of software, instead of only writing a AUTO232 driver, which is far easier and costs far less? 3. As I graduated in commercial economics at the Highschool for Business Economics, I have learned that a commercial organisation only makes an investment if they expect profit out of it: It should pay itself back, or better: The investment should make more profits. A lot of speculating has been done on this subject, but coming to the main point of this topic I would like to ask all the programmers on CCC the following questions: a) Is it possible that an autoplayer can influence other programs? I.e. overruling data, stopping some necessary routines etc. b) Is it possible to make an autoplayer that is advantageous for one side? I.e. makes the results for one side better than they really are. I want to point out that I am absolutely no expert on this subject, therefore I ask the authors of chess programs to give answers to these questions. The third question: c) If the answer to a) and b) is 'YES', would you agree to have your program tested against such autoplayers? I have the feeling that it was wrong from SSDF to accept this way of testing, WITHOUT consulting other programmers if they agree or not. A great risk is that now programmers are removing the AUTO232 software, or 'worse': are starting to make their own autoplayer. This leads to 6 different autoplayers (or more) and the results of SSDF would be getting more and more unclear. Furthermore: Instead of taking time and efforts to improve the playing strength and features of a program, all programmers have to spend a huge amount of time in autoplayers, booktuning, booklearners and so on. Inevitable conclusion: The real playing strength improvements are becoming less and less, instead a lot of 'statistical imrpovements' are made. F.e. who is the best in copying won games to gain more Elo-points. (Note that older programs do not have any defence to such treatments. If you take program A without learner and the very same program with a learner, than the last one will win, although it is the same program and absolutely not stronger at all. But statistically everybody will say that the second program is stronger). To make an end to speculations, above mentioned developments and the recent 'bookwar' (as I call it myself), I would like to come up with two proposals of new test methods, that could be used to determine the real differences in playing strength between chess programs. I do not want to imply these tests are the 'coup the grace', but still it might be useful as an alternative, to prevent the influence from killerlines, booklearners and so on. Proposal 1: The universal SSDF openingbook ------------------------------------------------------------------ * Out of - let's say 500,000 - grandmaster games the SSDF makes a universal openingbook. * This openingbook will not be published or made available to the chess programmers. * All tested programs will have to use the universal book, all testgames and matches will be played only with this book. * Each match consists of a predefined number of games, let's say 40. All matches should consist of 40 played games. * Doubles are not counted. * The configuration should be exactly the same for both opponents. A standard can be agreed upon, f.e. Pentium 200 MMX with 64 MByte of memory. Under these conditions older programs can participate as well, as the results of improved openingbooks, bookkilling, and booklearners are ruled out. Although a disadvantage could be that the new programs - with learners - could learn which lines in the new book are suitable and which are not. One can argue that a universal openingbook violates the fact that the openingbook is a part of the chessprogram. I agree to that, but I want to point out that this testmethod is meant to compare playing strength of chess programs, not opening books. Proposal 2: 50 openingpositions, each program plays one game with White and one with Black in every openingposition. ------------------------------------------------------------------------------------------------------------------- In Computer Schach & Spiele (CSS) the Nunn testpositions were introduced, as well as the SeE-test (see f.e. the excellent article in the latest CSS by Arno Nickel, page 45-49). Some six years ago I had been making a comparable test- set, to test the several ChessMachine versions of TASC (e.g. The King and ChessMachine Madrid). IMO the current Nunn- and SeE-tests are a very interesting approach to compare playing strength of chessprograms, the disadvantage being that these two tests contain too few testpositions. In the ChessMachine-period my test used to have 50 testpositions, containing 50 different testpositions following the next rules: * All popular and theoretically interesting openinglines are in this test * Less popular lines are also included * The test contains open, closed and semi-closed positions, as well as positions with isolated pawns, blocked structures etc. * The endposition of each test should not give one side a clear advantage, to prevent 1-1 results in that line all the time. * In a match both programs play each testposition as White and as Black. The advantages are clear: Booktuning, booklearning and other stuff are out of the question and because a valid testrange is used, containing positions that appear often in tournaments between strong players, such a test will provide good information about the real playing strength of a chess program. It is also possible to compare different versions of the same programmer, f.e. matches between MCP 5,6,7 Genius 3,4,5 Fritz 3,4,5 and Hiarcs 4,5,6 can be held. This way you can see in which way a program has been improved (or not). Of course there are disadvantages as well: 100 games 40/2 hours take a lot of time. Before it is possible to make a new Elo-list following this testmethod it will probably take a year or something.... To make it possible we could make the next rules: 1. The 50 positions are from modern theory and will be chosen by a strong chess player. 2. These positions will not be published or given to one of the competitors. 3. Each match the both competing programs have to play each position as White and as Black. 4. Openingbooks are turned off, all games have to start from the testposition. 5. At least 5 different matches (500 games) have to be played before a program appears on the ratinglist. This way we can provide a ratinglist that is free from influences caused by booktricks, giving a good example of the allround playing strength of a chessprogram. Also statistics can be made which positions a program plays well and which not. (See also Arno Nickel's article in CSS). At last I want to present a possible test of 50 testpositions, just to give an example how I made them 6 years ago for testing the ChessMachine. (see below) I want to stress the fact that sometimes people forget what an enormous job our chess friends in Sweden are doing for the sake of computer chess. This is all done by volunteers, free of charge and devoting parts of their lives just to give people insights in the rankings of chess programs. A great reward should go to them, because nobody asked for this Elolist and still they are testing chess programs, for the benefit of programmers, chess enthousiasts and people who are needing information for buying a chess program. In the meantime, however, it is also necessary to ask ourselves whether the recent developments are in the interest of computerchess or not. Maybe other testmethods are necessary. I would like to ask all people involved to think about this idea. All suggestions are welcome. 50 testpositions to test playing strength of chess programs =========================================== Position 1: Ruy Lopez (1) ------------------------------------- 1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. 0-0 Be7 6. Re1 b5 7. Bb3 d6 8. c3 0-0 9. h3 Bb7 Position 2: Ruy Lopez (2) ------------------------------------ 1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. 0-0 Nxe4 6. d4 b5 7. Bb3 d5 8. dxe5 Be6 Position 3: Ruy Lopez (3) ------------------------------------ 1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Bxc6 dxc6 5. d4 exd4 6. Qxd4 Qxd4 7. Nxd4 Position 4: Sicilian (1) ------------------------------- 1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Be2 e6 7. 0-0 Be7 8. f4 0-0 9. Be3 Qc7 Position 5: Sicilian (2) ------------------------------- 1. e4 c5 2. Nf3 Nc6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 d6 6. Bg5 e6 7. Qd2 a6 8. 0-0-0 h6 9. Be3 Be7 10. f4 Position 6: Sicilian (3) ------------------------------- 1. e4 c5 2. Nf3 e6 3. d4 cxd4 4. Nxd4 Nc6 5. Nc3 a6 6. Be2 Qc7 Position 7: Sicilian (4) ------------------------------- 1. e4 c5 2. c3 d5 3. exd5 Qxd5 4. d4 Nf6 5. Nf3 Bg4 6. Be2 e6 7. 0-0 Nc6 8. h3 Bh5 9. Be3 cxd4 10. cxd4 Be7 11. Nc3 Qd6 Position 8: French (1) ------------------------------- 1. e4 e6 2. d4 d5 3. Nc3 Bb4 4. e5 c5 5. a3 Bxc3 6. bxc3 Ne7 7. Nf3 Position 9: French (2) ------------------------------- 1. e4 e6 2. d4 d5 3. Nd2 c5 4. exd5 exd5 5. Ngf3 Position 10: French (3) -------------------------------- 1. e4 e6 2. d4 d5 3. e5 c5 4. c3 Nc6 5. Nf3 Qb6 6. a3 c4 Position 11: Caro-Kann (1) -------------------------------------- 1. e4 c6 2. d4 d5 3. Nc3 dxe4 4. Nxe4 Nd7 5. Ng5 Ngf6 6. Bd3 e6 7. N1f3 Bd6 8. Qe2 h6 9. Ne4 Nxe4 10. Qxe4 Nf6 11. Qe2 Position 12: Caro-Kann (2) -------------------------------------- 1. e4 c6 2. d4 d5 3. Nc3 dxe4 4. Nxe4 Bf5 5. Ng3 Bg6 6. h4 h6 7. h5 Bh7 8. Nf3 Nd7 9. Bd3 Bxd3 10. Qxd3 e6 11. Bd2 Qc7 12. 0-0-0 Ngf6 13. Ne4 0-0-0 14. g3 Position 13: Caro-Kann (3) ------------------------------------- 1. e4 c6 2. d4 d5 3. e5 Bf5 4. Nf3 e6 5. Be2 Position 14: Scotch ---------------------------- 1. e4 e5 2. Nf3 Nc6 3. d4 exd4 4. Nxd4 Bc5 5. Be3 Qf6 6. c3 Nge7 Position 15: Italian game ----------------------------------- 1. e4 e5 2. Nf3 Nc6 3. Bc4 Bc5 4. d3 Nf6 5. c3 d6 6. b4 Bb6 7. a4 a6 Position 16: Petroff --------------------------- 1. e4 e5 2. Nf3 Nf6 3. Nxe5 d6 4. Nf3 Nxe4 5. d4 d5 6. Bd3 Nc6 7. 0-0 Be7 Position 17: Four Knights ------------------------------------ 1. e4 e5 2. Nf3 Nc6 3. Nc3 Nf6 4. Bb5 Bb4 5. 0-0 0-0 6. d3 d6 7. Bg5 Bxc3 8. bxc3 Position 18: Pirc-Ufimzew (1) ------------------------------------------ 1. e4 d6 2. d4 Nf6 3. Nc3 g6 4. f4 Bg7 5. Nf3 0-0 6. Bd3 Position 19: Pirc-Ufimzew (2) ------------------------------------------ 1. e4 d6 2. d4 Nf6 3. Nc3 g6 4. Nf3 Bg7 5. Be2 0-0 6. 0-0 Bg4 Position 20: Scandinavian ------------------------------------- 1. e4 d5 2. exd5 Qxd5 3. Nc3 Qa5 4. d4 Nf6 5. Nf3 c6 6. Bc4 Bf5 Position 21: Alekhine ------------------------------ 1. e4 Nf6 2. e5 Nd5 3. d4 d6 4. Nf3 g6 5. Bc4 Nb6 6. Bb3 Bg7 7. a4 a5 Position 22: Modern Defence ------------------------------------------ 1. e4 g6 2. d4 Bg7 3. Nc3 d6 4. f4 c6 5. Nf3 Bg4 6. Be3 Qb6 7. Qd2 Nd7 Position 23: Slav (1) ----------------------------- 1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 dxc4 5. a4 Bf5 6. e3 e6 7. Bxc4 Bb4 8. 0-0 Nbd7 Position 24: Slav (2) ----------------------------- 1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 dxc4 5. a4 Bf5 6. Ne5 e6 7. f3 Bb4 8. e4 Bxe4 9. fxe4 Nxe4 10. Bd2 Qxd4 11. Nxe4 Qxe4 12. Qe2 Bxd2+ 13. Kxd2 Qd5+ 14. Kc2 Na6 15. Nxc4 Position 25: Slav (3) ----------------------------- 1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 e6 5. e3 Nbd7 6. Bd3 dxc4 7. Bxc4 b5 8. Bd3 Bb7 9. 0-0 a6 10. e4 c5 Position 26: Slav (4) ----------------------------- 1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 e6 5. e3 Nbd7 6. Qc2 Bd6 7. Bd3 0-0 8. 0-0 Position 27: QGD (1) ------------------------------ 1. d4 d5 2. c4 e6 3. Nc3 Nf6 4. Bg5 Be7 5. e3 0-0 6. Nf3 h6 7. Bh4 b6 Position 28: QGD (2) ------------------------------ 1. d4 d5 2. c4 e6 3. Nc3 Nf6 4. cxd5 exd5 5. Bg5 Be7 6. e3 c6 7. Qc2 0-0 8. Bd3 Nbd7 9. Nge2 Re8 10. 0-0 Nf8 Position 29: QGD Tarrasch --------------------------------------- 1. d4 d5 2. c4 e6 3. Nc3 c5 4. cxd5 exd5 5. Nf3 Nc6 6. g3 Nf6 7. Bg2 Be7 8. 0-0 0-0 9. Bg5 cxd4 10. Nxd4 h6 11. Be3 Re8 Position 30: QGD Semi-Tarrasch ----------------------------------------------- 1. d4 d5 2. c4 e6 3. Nc3 Nf6 4. Nf3 c5 5. cxd5 Nxd5 6. e4 Nxc3 7. bxc3 cxd4 8. cxd4 Bb4+ 9. Bd2 Bxd2+ 10. Qxd2 Position 31: QGA (1) ------------------------------ 1. d4 d5 2. c4 dxc4 3. Nf3 Nf6 4. e3 e6 5. Bxc4 c5 6. 0-0 a6 7. a4 Nc6 8. Qe2 cxd4 9. Rd1 Be7 10. exd4 0-0 11. Nc3 Position 32: QGA (2) ------------------------------ 1. d4 d5 2. c4 dxc4 3. e4 e5 4. Nf3 Bb4+ 5. Nc3 exd4 6. Qxd4 Qxd4 7. Nxd4 Nf6 8. f3 Position 33: Nimzo-Indian (1) ----------------------------------------- 1. d4 Nf6 2. c4 e6 3. Nc3 Bb4 4. e3 0-0 5. Bd3 d5 6. Nf3 c5 7. 0-0 Nc6 8. a3 Bxc3 9. bxc3 Position 34: Nimzo-Indian (2) ------------------------------------------ 1. d4 Nf6 2. c4 e6 3. Nc3 Bb4 4. Qc2 0-0 5. a3 Bxc3 6. Qxc3 b6 7. Bg5 h6 8. Bh4 Bb7 Position 35: Queen's Indian (1) -------------------------------------------- 1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Ba6 5. b3 Bb4+ 6. Bd2 Be7 7. Bg2 d5 8. Ne5 Nfd7 9. Nxd7 Nxd7 10. Bc3 0-0 11. Nd2 Rc8 12. 0-0 Position 36: Queen's Indian (2) -------------------------------------------- 1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. a3 Ba6 5. Qc2 Bb7 6. Nc3 c5 7. e4 cxd4 8. Nxd4 Position 37: King's Indian (1) ----------------------------------------- 1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 0-0 6. Be2 e5 7. 0-0 Nc6 8. d5 Ne7 9. Ne1 Nd7 10. Nd3 f5 11. Bd2 Nf6 12. f3 f4 Position 38: King's Indian (2) ----------------------------------------- 1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 0-0 6. Be2 e5 7. 0-0 Nc6 8. d5 Ne7 9. b4 Nh5 10. Re1 Position 39: King's Indian (3) ----------------------------------------- 1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. f3 0-0 6. Be3 e5 7. d5 Nh5 8. Qd2 f5 Position 40: Grunfeld Indian ---------------------------------------- 1. d4 Nf6 2. c4 g6 3. Nc3 d5 4. cxd5 Nxd5 5. e4 Nxc3 6. bxc3 Bg7 7. Bc4 c5 8. Ne2 Nc6 9. Be3 0-0 10. 0-0 Position 41: Benoni ---------------------------- 1. d4 Nf6 2. c4 c5 3. d5 e5 4. Nc3 d6 5. e4 Be7 Position 42: Benko gambit -------------------------------------- 1. d4 Nf6 2. c4 c5 3. d5 b5 4. cxb5 a6 5. bxa6 Bxa6 6. Nc3 g6 7. Nf3 Bg7 8. g3 d6 9. Bg2 0-0 10. 0-0 Nbd7 Position 43: English (1) --------------------------------- 1. c4 e5 2. Nc3 Nf6 3. Nf3 Nc6 4. g3 Bb4 5. Bg2 0-0 6. 0-0 e4 7. Ng5 Bxc3 8. bxc3 Re8 Position 44: English (2) --------------------------------- 1. c4 Nf6 2. Nf3 e6 3. Nc3 Bb4 4. Qc2 0-0 5. a3 Bxc3 6. Qxc3 c5 7. g3 b6 8. Bg2 Bb7 Position 45: English (3) --------------------------------- 1. c4 c5 2. Nc3 Nc6 3. Nf3 Nf6 4. g3 g6 5. Bg2 Bg7 6. 0-0 0-0 7. d4 cxd4 8. Nxd4 Nxd4 9. Qxd4 d6 Position 46: Reti ------------------------ 1. Nf3 d5 2. c4 c6 3. g3 Nf6 4. b3 Bg4 5. Bg2 e6 6. Bb2 Nbd7 7. 0-0 Bd6 8. d3 0-0 9. Nbd2 Position 47: Bird's Defence --------------------------------------- 1. f4 d5 2. Nf3 Nf6 3. e3 Position 48: Dunst/Van Geet opening ------------------------------------------------------ 1. Nc3 Position 49: Flank opening -------------------------------------- 1. g3 e5 2. Bg2 d5 3. d3 Position 50: Larsen's opening ------------------------------------------ 1. b3 e5 2. Bb2 Nc6 3. e3
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.