Author: pavel
Date: 08:38:53 12/18/01
Go up one level in this thread
On December 18, 2001 at 11:08:47, José Carlos wrote: >On December 18, 2001 at 10:19:05, pavel wrote: > >>On December 18, 2001 at 09:52:49, José Carlos wrote: >> >>>On December 18, 2001 at 08:20:52, pavel wrote: >>> >>>>(Arguably) The strongest commercial chess program vs (Arguably) the strongest >>>>freeware chess program, in a very arguable matchup. >>>> >>>>;) >>>> >>>>-------------------------- >>>>Book = 2600.ctg >>>>Hash = 50mb both >>>>TB = none. >>>>Time Control = 5min/side >>>>Ponder = off >>>>Hardware= Pentium III/ 512mb ram. >>>>OS = Windows 2000 Pro. >>>>--------------------------- >>>> >>>> >>>> >>>> Program Elo + - Games Score Av.Op. Draws >>>> >>>> 1 Fritz 7 : 2580 36 58 200 71.5 % 2420 21.0 % >>>> 2 Crafty 18.12 : 2420 58 36 200 28.5 % 2580 21.0 % >>>> >>>> >>>> >>>>Individual statistics: >>>> >>>>(1) Fritz 7 : 200 (+122,= 42,- 36), 71.5 % >>>> >>>>Crafty 18.12 : 200 (+122,= 42,- 36), 71.5 % >>>> >>>> >>>>(2) Crafty 18.12 : 200 (+ 36,= 42,-122), 28.5 % >>>> >>>>Fritz 7 : 200 (+ 36,= 42,-122), 28.5 % >>>> >>>> >>>> >>>>The differance between freeware chessprograms and commecial programs seems to be >>>>just going bigger. Ok, Ok probably this result doesnt say much but, I am sure >>>>this is the case. Or is it that arguable? >>>> >>>> >>>>Have fun, >>>>Pavs. >>> >>> Let's be scientific. Your test shows: >>> >>> Fritz 7 + 2600.ctg >>> seems stronger at 5 min/game in a PIII unknown mhz + ponder off than >>> Crafty 18.12 + 2600.ctg >>> with a certain degree of confidence given by the number (200) of games. >>> >>> Neither program use their default book. The time/move is unknown since we >>>don't know the clock speed. Ponder is off which is not a default setting. >>> >>> I don't know your test is worthless, don't get me wrong. I only say it does > > Sorry here, my horrible english... I meant: "I don't mean your test is > ^^^^ >worthless, don't get me wrong." > >>>not prove anything but the above stated. Nothing about commercials or amateurs; >>>fritz or crafty; fritz or crafty + default settings; and so on... >>> >>> José C. >> >> >>oh yeah ofcourse I forgot to put, Pentium III 1Ghz. >> >>Even though I am not going to try to say that my test is the best. but probably >>is not worthless either. > > As I say above, I don't think it either. > >>1) POnder off is a default setting under CB interface. Since both programs are >>not pondering, I dont see a problem > > Problem is that Bob has stated many times 'his default' is ponder on. So >ponder off is not default for crafty, so in some way, it hurts its strength. Yes I have seen it discussed here many times before. And to be honest, I was against, crafty playing with ponder=off, for a long time. But the problem is there is no valid data (even though I understand the author insists) to prove that it effects the playing strength of the program. > >>2)Both program used same opening book from a well-known set of pgn file. If >>there is anything wrong with the opening book, both program will suffer. As the >>opening is reversed in every game. IMO the strength of the program doesnt >>include opening book, opening book is a way to increase the strenght of a >>program. > > This has been discussed many times, so maybe I should bring it up again but I >can't resist :) > The book is part of the program. Different books make the program play >different positions. If you use a book with very positional lines in a Hiarcs - >GT match it will probably benefit Hiarcs. If you use a wild book, it will >probably be better for GT. > In both cases the book is the same for both programs, but the result is quite >different. > The book, as the rest of the program, has a 'style'. For example, I'm working >on a tournament book for my program for several months. I don't only chose >'correct' lines, but lines where my program play correctly. I've found many pawn >sacs in GM's games that make my program instantly show -0.90. I don't want such >lines in my book even if they're correct... but GT would probably love them... Interesting, I agree that the book does define the playing strength of a program. But the question in hand is, if two programs not playing with their default book, shouldnt that effect both the program playing strength? Considering the fact the opening is reversed for both the programs. > >>It is a well-known fact in this forum, that you can never be perfect in a >>eng-eng match. No matter how many games you play or whatever precautions you >>take. > > Sure. And I have no problem about it, since it happens to all of us. I only >have 'problems' (not really problems... simply I disagree) with incorrect claims >about the meaning of the matches. > >>Even though the games were just fun, i was just trying to get some meaning out >>of it. > > Yep, that's the problem. Getting meaning out of games is difficult and >'dangerous'. > I'll tell you a little story: when I first read a post of Christophe claiming >that a lost games is worthless for him I thought his was just disappointed for >losing. Later, I rewrote his words many times and understood his point. The >point is: you modify something in your program in order to get a better >performance over a lot of games, not in order to correct something in >particular. That way, you get better for sure. So, it's something like a >'quantum chess'. A single game (particle) means nothing. It's a big number of >them that make sense. And not only that: the meaning depends on the >'circumstances' how the games were played. > > >>regards >>;) >>pavs What do you suggest would be a more "fair" match up between these 2 programs? I will state few ideas, pls let me know what you think 1) Default Opening Book for the programs. (problem here, does crafty have a default opening book? the one thats available for download, is being used for a lot of versions, and not sure if it has been tweaked for craftys style of play. Even though lets say we want to use that book, we cant use it under CB interface AFAIK. So we will have to use crafty under winboard in one computer and make it play against fritz7 with its own GUI on another computer.) 2) Ponder=on 3) 2 Identical CPUs (thats an obstacle I cant overcome, gotta buy another computer.) 4) Tournament Time Control 5) reasonable amount of game, 500. IMO thats reasonable enough. regards. pavs > > Regards, > > José C.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.