Author: Sune Fischer
Date: 15:04:53 09/12/04
Go up one level in this thread
On September 12, 2004 at 17:39:26, Robert Hyatt wrote: >>I don't believe it's just one big fuzzy thing that can't be seperated. > >I consider a "computer chess playing entity" to be just that. The sum of _all_ >the parts. If you want to test individual parts, fine by me. But what you are >testing has _nothing_ to do with how the complete "entity" plays chess. I won't go too much into this again, I think Sandro and I already have discusses this :) > It >won't predict how well it analyses. Why not? You mean to say that giving it a very strong book or very weak book will make it _easier_ to see how strong it is in analysis? That makes absolutely no sense at all. >How well it will do against humans or >computers. Or anything else... No but it will reveal its strong and weak points which might be of interest to the user. >> >>>Why does it matter how Fritz does with a bad book? >> >>Suppose we gave the Fritz book to a strong amateur engine, Aristarch for >>instance and in a very long match it beated Fritz. >> >>That would obviously be interest for several reasons. > > >No more so than if someone builds a new book from a PGN collection and produces >the same kind of result. If the goal is to find the best book for program X, >then such a test would make sense. But that is _not_ the goal that is being >discussed. You are the one who has been saying it makes _no_ sense to play without own books, I'm just trying to show you that there are reasons to play without them. My job is easy, I just need one single counter example :) >It is taking hokey positions, making programs play them against each >other, and then trying to draw conclusions from that. The two are _not_ the >same thing. > >Ditto for learning on/off, pondering on/off, etc... I disagree. >>>No endgame tables? >> >>There is no room for endgame tables on his laptop. > >Baloney. I have a sony VAIO with a 20 gig hard drive. I have _all_ the 3-4-5 >piece files on it... 20 gig drives are small today. I have a 10 GB drive and it is full. To take another example, how are you going to use endgame tables on the PocketPC?` http://www.pocketgear.com/software_detail.asp?id=15142 >> >>>Impossibly short time controls? >> >>He needs to analyse 50000 games. > >For what possible reason that makes any sense??? Ask him, I won't be the judge of what people should and shouldn't do. >>> No pondering? >> >>He has a single CPU machine. > > >So? I do ponder=on matches on my single-cpu laptop all the time. No problems >at all How do you make sure they get 50% cpu each? What happens when one engine hits ETGB or runs a high priority thread? >>> No learning? >> >>He wants reproducable results. > >He wants meaningless results you mean. I can believe the low regard you hold on reproducability, it is just the foremost important property of any experiment. How do you measure progress without reproducability? Say he wants to see how much changing the hash size means for Crafty - he can't conclude anything due to the learning. Say he changes some evaluation parameters and wants to see if Crafty plays better - he can't conclude anything due to the learning. etc... >Suppose one person hand-tunes their >book. The other chooses to go the book-learning route instead. This test is >therefore flawed in a most basic way. Not in testing analysis power. >> >>>Why not test with "no code" as well??? >> >>He already knows how strong that would play. > > >Apparently not. You forgot a smiley :) >> >>>>Suppose the book is worth 100 Elo and Fritz is the only one who is allowed to >>>>use that book, now obviously Fritz will look 100 Elo stronger in all matches >>>>than it really is, and obviously these 100 Elo are worth nothing to a >>>>correspondence player who only needs the engine for analysis. >>> >>>Au Contrare, Fritz will be giving _good_ opening advice, for one thing... >> >>I think the GUI+book will be doing that. > >So? That is, by definition, "Fritz". Ehmm, so it is the same program competing several times in WCCC? >> >>>And if you expect _any_ program to give good advice on oddball openings, good >>>luck... >> >>I expect a program do the best it can, even in objectively lost situations. >>There is honour in fighting for a draw as well :) >> >>-S. > >Certainly, but I don't plan on testing in every possible kind of position. I >just avoid the ones that don't look particularly reasonable and leave it at >that. It works... It works for playing a quick tournament. For the long run development and to be strong in general analysis I think it is interesting to investigate and improve the weak points also. -S.
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.