Author: Uri Blass
Date: 08:35:08 02/16/02
Go up one level in this thread
On February 16, 2002 at 10:32:27, Thorsten Czub wrote: >On February 16, 2002 at 09:37:31, Bertil Eklund wrote: > >>On February 16, 2002 at 09:23:16, Torsten Schoop wrote: >> >>>A1200 vs. K6-450 is nonsense IMOP. >>> >>>Torsten >> >>Yes it is nonsense to play in tournaments with players 150 elo above or under >>yourself too. >> >>Bertil > >you don't understand the difference between computerchess and humanchess. > >in a tournament, it makes sense to play against weaker or stronger PLAYERS. > >it makes even sense to play in a tournament one or 2 games >shredder6 1200mhz versus shredder6 450 mhz. > >but it makes NO sense , and here torsten is IMO right, to try a match >program X versus program X' with X on 1200 and X' on 450mhz. > >chess programs are not players in a common sense. >they are a special form of players. a subgroup. > >because they produce nearly similar results when the search depth >is similar. > >so giving program X 6 plies and giving program X' 8 plies will make >X' win. Not always and the results of the games showed that Shredder6 did not get 100% against shredder5.32 on inferior hardware I agree that shredder6 won rating points from the games against shredder5.32 on inferior hardware but it is not always the case with chess programs. There are some possible reasons for the newer program to lose or draw against the old program: 1)It is possible that there are positions when the knowledge of the old program is superior. Even if the new program is better it does not mean that you improve it in every position and there may be games when the newer version is unlucky to get positions that it does not understand. 2)bad choice of the opening by the new program. It is possible that the older program got a position that it can win only because of book or position that is drawn only because of book. 3)some luck It is possible that the new program outsearches the old program and win material only to find in the end that it is losing because of something that was too deep for both programs in the beginning of the combination. > > >human players do not think SIMILAR since the human brain is a quantum >mechanic kind of computer. > >so even when you have a TWIN , say bobby fischer against his brother (if he >would have a twin brother) it would NOT be as similar and clear as if you have 2 >programs. > >Even if you would clon bobby fischer, his brain would still be a quantum >mechanics kind of computer brain. and this is NOT the same as a primitive >computer where every event can be repeated. > >the program X is so similar to program X' that the results between them >is NOT showing a real playing strength difference. > >the same problem you have when you decrease time controls. say, you let >n computer-programs play y games against each other in a time control >of 1' or 2' or even 5' per game blitz. > >call this engine tournament a blitz tournament. >you will NOT get senseful results, because you do not measure >the playing strength of the chess programs, but instead you measure how >fast the search tree of the programs is build to come deeper. By this logic games between chess program more than 10 years ago on some old 386 measured only how fast the search tree of the program is build. I do not agree. I think that search is very important in computer chess but it is not the only factor and if you have 2 programs that search in the same speed when one of them has better evaluation then the program with better evaluation is going to win in blitz. > >so in fact you measure not chess strength but the WAY the chess strength >is build. > >and the problems get even bigger when the user-interface you want >to test the FAIR engine competition (lets say i am using a chess-base >interface like fritz, junior, nimzo or whatever GUI) >is NOT giving all engines the same resources of the pc. > >then in fact you can play a big engine tournament, and most often >the people play short time controls to get many games :-))) >that in the end, as a result the whole engine-tournament >is nearly nonsense. because the NOISE that gets produced by >the circumstances taking action in the measurement, that you >don't get the real strength, but instead a noisy up and down >of results. > >this is the opposite of beeing scientific. thats making an idiot out >of the reader of such a statistic. > >if you want to find out about strength, you must make sure the competition >is fair. > >that means the results have to be reproducable. >and you have to protocol the games. >and the machines have to be identical. >and double or tripple games have to be deleted out of the statistic, >you must make sure, in general spoken, that the amount of noise >that tries to influence the result is as small as possible. > >what worries me much is that shredder was unable to win a tournament >where Mr.Utzinger or his friend played, and shredder was on the other >hand able to reach the top of your list. > >there is something stinking in the fact that a program >is making number ONE in matches in sweden, >and making the last rank in a tournament in austria. > >i don't believe in this heavy dispersion. > >one of you has a bug or a problem somewhere. > >since mr.utzinger played on ONE machine, it cannot >be that he used non-identical resources. >and since the ranking of his other programs and the >games seem to be normal , from my point of view, >the reason why shredder6 won in sweden >can only be in the >learning >or in the permanent brain taking influence. > >or - in the autoplayer :-)) or maybe we have not enough games The difference between shredder and the other programs was very small and part of the players in mr utzinger tournament(Fritz7.002,the king with special setting) did not play in the ssdf games. In the case of Fritz I guess that they did not want the ssdf to test a version with the castling bug and I guess that Fritz7 is going to be against number 1 in the next list because I guess that tiger15 is not going to be in the next list. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.