Author: martin fierz
Date: 14:49:51 03/30/02
Go up one level in this thread
On March 30, 2002 at 16:57:48, Slater Wold wrote: >On March 30, 2002 at 16:02:17, martin fierz wrote: > >>>Asking a GM to play a game, and then asking for 2 or 3 times more time wouldn't >>>go over very well. >>huh? of course not. but a computer running twice as fast gets the same result in >>half the time. you can't compare that to what a human does! >> >>>Running Nolot for 48 hours to get a 24 hour result isn't something I am >>>interested in. >>why? you will get the same result as if your program were twice as fast. if all >>you want to know is what a twice as fast crafty would solve in nolot, then you >>can answer this question right here, right now, without FPGA. >> >>>HW based move ordering is *always* more efficent and cannot usually be >>>reproduced with software under any circumstance. >>what can you do with the hardware that you can't do in software (except that >>it's much slower)? of course it is more efficient when it's faster. >> >>>There is a LOT that changes with a program when one of it's heaviest pieces are >>>done in almost 0 time. Changes you cannot reproduce with more time. >>like what? i really don't get it :-) of course it will be a better program, but >>i fail to see how this relates to the 3 goals you set above. it's not that i'm >>saying you shouldnt do this, because it's really interesting - just for other >>reasons than you give IMO. >> >>aloha >> martin > >These are my thoughts. Agree, disagree, it's up to you. ;) that's the spirit! :-) >In Belle's days, it was widely agreed that if a *good* chess program could hit >14 ply per move in a game, a GM would never stand a chance. This is *obviously* >not true. There are engines out there that still only do 8-10 ply in 3 minutes, >but there are some that do 15-18. And computers still do not rule chess. Why? > >Well, most people say it is because there is an effective range for all engines. > Lets say HIARCS 7.32 effective range is 8-10 ply while Junior 7's is 15-18 ply. > Anything less than these ply and the engines will be a *lot* weaker. Anything >more than these ply and the engines will not gain much. It is obvious that ply >== Elo. Why else does it still take engines an hour to find certain moves that >are obvious to GMs? I say it's obvious, others don't think so. i don't believe in this effective range theory. that sounds like an excuse that the guys with the weak engines make :-) i have tested my checkers program in self-play mode with different search depths against a version with higher search depth, and over the whole depth range investigated, the "faster" program is better - with diminishing returns, but always better. in chess, the point of diminishing returns is not reached AFAIK. >Of course if you give an engine extra time on a test suite, it will usually get >a few more answers. But how does that relate to PLAYING performance? Does it >even relate to playing performance. A while back, I was attempting to run a NPS >challenge pitting Crafty vs Rebel. I played 3 games at 40/120 and Rebel won >*every* game. However, how would Rebel do if I gave Crafty 100x more time? (I >never got to that part because Ed never sent me Rebel 4 and I ran out of "trial >time".) Playing a time odds match computer vs computer would show you a *good* >starting indication, but when it came to playing GMs, this would show nothing. even deep blue lost a game against fritz (maybe it was called deep thought then) when it played into a lost position out of book, or made an error on the first move out of book (it was a sveshnikov, DB white, got under big attack on the kingside). and 3 games prove nothing, of course. my self-play tests were done with 282 games per match, and without opening books. like this, you really measure the engine strength, and not if a program has a better book (as was the case in the fritz-DB game). and with 282 games you get a statistical uncertainty of only about 2% on the resulting win-percentage, which is good enough to show the diminishing returns. >So here is my thinking: > >1.) Crafty on my computer (Dual AMD 1.6Ghz) reaches a certain Elo esitmate. How >do I make it faster? Giving it more time would start to show a good esitmate, >but it's still not a "real world" figure. > >2.) I could compare it to a slower machine. However, then people start saying >that Crafty is not reaching its "optimal depths" and therefore will of course >play weaker. even though i don't believe in the optimal depth thing, i guess if you go way out of the design range, like 100 times slower, then it really won't perform efficiently. i think you are right not to do this. >3.) We can prove time and time again that Crafty at 12 ply is stronger than >Crafty at 9 ply. But how do we prove in REAL TIME that Crafty at 15 ply is >stronger than Crafty at 12 ply? i don't think you need to prove this. it's about like proving 15>12 :-) >There really is only one way. Speedup the hardware. > >If I only speedup Crafty by 50% then perhaps using more time would be a >practical way to do the same experiment. But what if I speedup Crafty by 500%? >Of course, at first I do no expect to see gains of 500%, hell, I would be happy >with 50%. But this is just the beginning. Crafty at 100M nps would be a *very* >interesting thing. And that is bascially my ultimate goal. aha! now you're talking :-) that was my point - if you only expect to get it to twice the speed, you can always test it with more time. but if you have set your sights to higher goals, and this is just a test to see how things go - then it is a different matter entirely, and i agree with you that it's impractical to test it in "real time" if that means that one game lasts a week! i ran my self-tests up to the point where the last match took one week to complete, and the next level would have been 1-2 months i guess, so i stopped there... aloha martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.