Author: Robert Hyatt
Date: 14:39:26 09/12/04
Go up one level in this thread
On September 12, 2004 at 10:47:02, Sune Fischer wrote: >On September 12, 2004 at 10:18:47, Robert Hyatt wrote: > >>>>These "matches" don't show _nearly_ as much as many believe... >>> >>>They show me what I want to know, ie. how good is Fritz _without_ the killer >>>book from chessbase? >> >>Why does it matter? > >I don't understand why this is such a big deal, a chess package consists of >several things like GUI/eyecandy, database/books, engine, server account etc. > >To some the database facilities might be very important while the look and feel >of the GUI is not so important, to some the book might be important and to >others the engine is the crucial component. > >I don't believe it's just one big fuzzy thing that can't be seperated. I consider a "computer chess playing entity" to be just that. The sum of _all_ the parts. If you want to test individual parts, fine by me. But what you are testing has _nothing_ to do with how the complete "entity" plays chess. It won't predict how well it analyses. How well it will do against humans or computers. Or anything else... > >>Why does it matter how Fritz does with a bad book? > >Suppose we gave the Fritz book to a strong amateur engine, Aristarch for >instance and in a very long match it beated Fritz. > >That would obviously be interest for several reasons. No more so than if someone builds a new book from a PGN collection and produces the same kind of result. If the goal is to find the best book for program X, then such a test would make sense. But that is _not_ the goal that is being discussed. It is taking hokey positions, making programs play them against each other, and then trying to draw conclusions from that. The two are _not_ the same thing. Ditto for learning on/off, pondering on/off, etc... > >>No endgame tables? > >There is no room for endgame tables on his laptop. Baloney. I have a sony VAIO with a 20 gig hard drive. I have _all_ the 3-4-5 piece files on it... 20 gig drives are small today. > >>Impossibly short time controls? > >He needs to analyse 50000 games. For what possible reason that makes any sense??? > >> No pondering? > >He has a single CPU machine. So? I do ponder=on matches on my single-cpu laptop all the time. No problems at all > >> No learning? > >He wants reproducable results. He wants meaningless results you mean. Suppose one person hand-tunes their book. The other chooses to go the book-learning route instead. This test is therefore flawed in a most basic way. > >>Why not test with "no code" as well??? > >He already knows how strong that would play. Apparently not. > >>>Suppose the book is worth 100 Elo and Fritz is the only one who is allowed to >>>use that book, now obviously Fritz will look 100 Elo stronger in all matches >>>than it really is, and obviously these 100 Elo are worth nothing to a >>>correspondence player who only needs the engine for analysis. >> >>Au Contrare, Fritz will be giving _good_ opening advice, for one thing... > >I think the GUI+book will be doing that. So? That is, by definition, "Fritz". > >>And if you expect _any_ program to give good advice on oddball openings, good >>luck... > >I expect a program do the best it can, even in objectively lost situations. >There is honour in fighting for a draw as well :) > >-S. Certainly, but I don't plan on testing in every possible kind of position. I just avoid the ones that don't look particularly reasonable and leave it at that. It works...
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.