Author: Sune Fischer
Date: 08:34:02 06/04/02
Go up one level in this thread
On June 04, 2002 at 07:53:24, Rolf Tueschen wrote: >>Yes, this is what we mean by indetermanism. >>Take the Zobrist table, if you do not initiate that with a random seed but use >>the clock generated one, then you will generate different keys at each search >>and so the hashtable saves different results and that will affect the search. >>Not _very_ random I guess, but not completely *determanistic* either. > >My knowledge about machines is not sufficient to debate that but I think that >this is strange. Why should a machine play at random if it once "thought" it has >found the best play. Would you say that even if the perfect way has been found, >that then still some random changes could have even better results? I don't get >this. You are getting too abstract for me here. "perfect play" i don't think that is a term that should be tossed around carelessly, it is very powerfull concept unachievable to us at the moment and for a long time to come. So unless the position is simple enough that a "perfect move" can be found (eg. a mate score), then yes the score could change. By randomness in the zobrist table, a mate-score from 1 search may be overwritten in a different search on the same position. So what the hashtable contains will depend on the keys, which in my program is generated at random. (hope that made some sense:) > >> >>>>GM's performances against computers recently have been rather dissapointing. >>>>Quite frankly, I am not sure who is exposing the weaknesses of the other more. >>> >>>Here my thought experiment could give you the right clue/ idea. >> >>Yes but we do not have the money to buy enough GM playing time needed for the >>10000 games, so it you can forget that idea I'm afraid. > >Excuse me, that was a fantasy of someone other. I don't think that we need 10 >000 games. What we need are opponents who don't play human chess against >machines. That's all. So you handpick the opponents and this would give a fair measure of strength? >>Say you take two football teams, France and Brazil and let them play eachother >>100 times. Say the score is 45-55, that gives us a measure of how much stronger >>Brazil is than France (if at all). This is "selfcalibration" if you want to call >>it that. It tells you nothing about how France would score against Germany, but >>for an internal reference in the pool it is very good. >>However if Germany played Brazil, the theory goes that we could use that result >>to pridict the outcome of France - Germany. >> >>We do not have GMs in the pool of computers, so the SSDF can only be used as an >>internal reference, noone disputes that I think. > > >We have a different understanding of calibrating then. Oh my God! >This is simply nonsense, excuse me so much. By definition you can't calibrate >France against Brazil and then claim that you are now thinking that Brazil is as >good as the gorillas in the jungle. Know what I mean? Actually no ;) What is it you claim besides Brazil being better than France (within some margin of error)? Are you trying to extract more information from the match? >As to your predictions, I agree that we could always be sure about the result in >matches between different "classes", say 2002 giants against 1999 winners. But >for 2202 giants for each other we do not know because of the ridiculously high >margins. That is statistics, the error bars will tell you how sure you are of the results. >And then a little reflection. Our predictions could also come out of games >itself, but not from the ranking list. That is my point. Of course only if >someone knows about computerchess and the games. If you have a concret proposal on how to do this, then that would be interesting. But lets say player A is completely dominating the board against B, totally outplaying B in positional play. Then A makes _1_ tactical error and lose the game. A and B play a lot of games and the final score is 50%-50%. Ok, lets say all games are like this, then who is stronger; A makes 1 very bad move every second game, B playes marginally worse all the time? I think any weakness is bad, and you need to be good all around to be good. The scores are the weights that measure the "badness" of you weaknesses. I don't see why a some subjective non-scientific method should be better. >>Like Dan said, it is not bogus but you need to understand what it does show. >>It cannot be compared to the FIDE scale, and nobody is saying it can. >>It may be close, we suspect that I guess, but we do not really know. > >Would you tell me why the numbers look so suspiciously close to some human >rankings? I mean if we take the performances of machines in show events. > >Hint, we're not talking about the surface of SSDF but about the "real" strength >of machines and humans. A lot of these machines are playing on internet chess servers against humans all the time, e.g. Crafty has played many GM's over the years. My highly subjective opinion is the that the SSDF is "close" to the FIDE rating, maybe errorbars of +-100 elo, I guess that is suspiciously close, but a rule of thumb is never to compare rating lists with each other (unless there are 20 identical players on both lists, then you can get an idea). >>>But what means Elo 2650 at SSDF??? Just this little question to begin with. Do >>>you have any idea? >> >>Well what does 2345 FIDE mean? - it is a number on a scale that is used to >>compare humans. > >I agree. And you would conclude that the SSDF numbers must be something similar? >Why? Where was this calibrated and where was is made valid? Lets say Program X is stronger than Program Y on the computer ratinglist, but as it happens Program Y is stronger against humans than X, then what? If we decide to let X and Y into the human pool, then X and Y will get new ratings based on this new data and possibly Y will now rank higher. This doesn't mean the old ratinglist was wrong, program X _was_ stronger than Y in the old pool. There is no conflict, unless you decide the ratinglist absolutely _must_ be indentical, that will create problems, obviously. The trick is - do not compare the lists :) -S.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.