Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: 100:1 NPS Challenge

Author: Robert Hyatt

Date: 09:27:16 12/19/03

Go up one level in this thread


On December 19, 2003 at 11:47:28, James T. Walker wrote:

>On December 19, 2003 at 00:26:41, Robert Hyatt wrote:
>
>>On December 18, 2003 at 16:18:21, James T. Walker wrote:
>>
>>>On December 18, 2003 at 16:07:44, Robert Hyatt wrote:
>>>
>>>>On December 18, 2003 at 15:53:02, James T. Walker wrote:
>>>>
>>>>>On December 17, 2003 at 13:20:03, Robert Hyatt wrote:
>>>>>
>>>>>>On December 17, 2003 at 13:09:57, Slater Wold wrote:
>>>>>>
>>>>>>>On December 17, 2003 at 10:41:24, Uri Blass wrote:
>>>>>>>
>>>>>>>>On December 17, 2003 at 10:23:26, Matthew Hull wrote:
>>>>>>>>
>>>>>>>>>On December 17, 2003 at 10:21:58, Slater Wold wrote:
>>>>>>>>>
>>>>>>>>>>On December 17, 2003 at 09:35:16, Robert Hyatt wrote:
>>>>>>>>>>
>>>>>>>>>>>On December 17, 2003 at 09:05:55, Slater Wold wrote:
>>>>>>>>>>>
>>>>>>>>>>>>I guess I will be running the 100:1 NPS challenge.  Here's the info:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>I will use any books.bin & bookc.bin that Bob asks me to.  The book.bin will be
>>>>>>>>>>>>created from enormous.pgn.
>>>>>>>>>>>
>>>>>>>>>>>My suggestion is to use book.bin, bookc.bin and books.bin from my ftp
>>>>>>>>>>>machine.  book.bin has no learning data so it will start off in the best
>>>>>>>>>>>possible way.
>>>>>>>>>>>
>>>>>>>>>>>remove position.bin before game 1.
>>>>>>>>>>>
>>>>>>>>>>>And, as I suggested previously, if, after a program leaves book, it is
>>>>>>>>>>>in an obviously won or lost position, the game gets aborted and the next
>>>>>>>>>>>one started.  There is no place for "book kills" when the goal is a time
>>>>>>>>>>>handicap match.
>>>>>>>>>>
>>>>>>>>>>Agreed.  The only loss Crafty has suffered in the Rebel match was a book loss.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>BTW, what were the results of that match?
>>>>>>>>
>>>>>>>>3.5-1.5 for crafty
>>>>>>>>
>>>>>>>>see http://www.talkchess.com/forums/1/message.html?336433
>>>>>>>>
>>>>>>>>Note that the match is not very interesting for me because it is an open
>>>>>>>>question if Crafty is better than rebel on equal hardware and in WBEC Crafty has
>>>>>>>>13/24 when Rebel has only 10/24
>>>>>>>
>>>>>>>There is no doubt in my mind Rebel is better than Crafty on equal hardware.  And
>>>>>>>I've played, oh, about 5,000 games with Rebel.
>>>>>>
>>>>>>I would take that wager.  We _both_ use quad opterons.
>>>>>>
>>>>>>:)
>>>>>>
>>>>>>Isn't that "equal" by any reasonable definition?  :)
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>How much better is questionable, but it's obviously not 8x.  ;)
>>>>>>>
>>>>>>>>The more interesting question is if Rebel is able to get better result than
>>>>>>>>Crafty in the premier division.
>>>>>>>>
>>>>>>>>Uri
>>>>>
>>>>>;)  They are not running on "equal" hardware if one is using 4 cpus and the
>>>>>other is using only one cpu.  I have several hundred blitz games of Crafty 19.7
>>>>>vs Rebel 12 on 2 XP2400+ machines/auto232.  I call that equal hardware.  In that
>>>>>case Rebel has a slight advantage on Crafty of maybe 30-40 Elo.  (According to
>>>>>the few hundred games so far).
>>>>>Jim
>>>>
>>>>
>>>>That is a bad definition of "equal hardware".  IE if two programs run on a
>>>>PIV 3.06ghz processor, but one uses SSE and the other doesn't, is _that_
>>>>also not equal?  Or one uses hyper-threading and the other doesn't?
>>>>
>>>>"equal hardware" means "platforms are identical".  What a program gets out
>>>>of those equal platforms is another matter.
>>>>
>>>>It takes effort to use that "extra stuff".  I played a couple of challenge
>>>>matches years ago when someone would say "Hey, you are using a Cray, if I
>>>>had something that fast, I could play equal to or better than you."  I had
>>>>them send me their code, I compiled and we played on the same machine, no
>>>>pondering, one cpu each.  What they overlooked was that I had invested a
>>>>lot of work getting the vector hardware to help me.  They hadn't.  So on
>>>>"equal hardware" I was 20x faster than they were and the match was not
>>>>that pretty.
>>>>
>>>>Doing a parallel search takes time.  Does it seem reasonable that my opponent
>>>>uses an extra year to improve his evaluation, while I use an extra year to get
>>>>a good parallel search done, then we say "your parallel search is an unfair
>>>>adevantage?"
>>>>
>>>>It's a different way of thinking about it when you think about it.  Those
>>>>extra CPUs don't just magically make the program faster without a _lot_ of
>>>>design effort and programming work.
>>>
>>>:)
>>>There is no denying that you have put in a lot of work on Crafty.  I and many
>>>others really appreciate what you have done.  That still does not make 1=4.  I
>>>wish I had a quad or even a dual to run the "deep" programs on but I don't.
>>>Maybe when the price comes down a little I can get something not quite on the
>>>leading edge that I can afford.  In the mean time, for me, 1=1 and 4 is 4x
>>>larger than 1. :)
>>>I said a few years ago that Crafty was showing the way for others to go in chess
>>>because I believed that CPUs have a practical upper limit of Ghz.  So eventually
>>>all will have to go to multiple cpu operation for more speed.
>>>Again I salute your work but it does not make 1=4. :)
>>>Happy Hollidays,
>>>Jim
>>
>>
>>Hint:  Hardware equality has _nothing_ to do with software.  Nor operating
>>system.  Nor the phase of the moon.  Hardware equality can be determined with
>>_no_ electricity whatsoever.
>
>That may be true but "Running on equal hardware" is not the same as haveing
>equal hardware available.  I get your point and I'm sure you get mine so again,
>Happy Holidays.
>Jim


I believe you mean "effectively using" rather than "running on". Anybody can
run on a Cray.  Not every program can effectively use it.  Those that do will
be 10x-100x faster than those that don't.  Yet they are _all_ "running" on
exactly the same processor.  It is definitely "equal" from the hardware
perspective.

You didn't answer my other point.  I spent a lot of time making my code use that
second processor.  A commercial programmer chose not to and spent more time
working on his search or evaluation.  If we both run on a single cpu machine,
that is _not_ equal in the context of your point above.  Because he is getting
more out of the single CPU than I do, because I spent less time there.

So trying to isolate true "equality" is not so easy.  Buy two of the fastest box
you can get, sit them side by side, and they are equal.  If a program can't use
all of what you bought, that still doesn't mean the two boxes are not equal,
just that the two programs are not equal.  Which could be true no matter _what_
hardware configuration you choose to use.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.