Author: Robert Hyatt
Date: 19:39:12 09/03/02
Go up one level in this thread
On September 03, 2002 at 21:18:39, Vincent Diepeveen wrote: >On September 03, 2002 at 21:08:39, Vincent Diepeveen wrote: > >When i did bob's test by the way, the problem was not so >hard to find why bob could never reproduce good speedups >with 1-8 cpu's. > >it is really very simple a permanent brain thing because >his test method sucked. What bob did was rerun whole >game in next way. > >he had written down in logfile where the 16 processor >search was and he had written down the search time needed >obviously. > >then he had for 1 processor it run to the same search depth. >then the right move was made of him and of the opponent and >the thing ran again on the same position. Nope. reread the test setup. This was an _automated_ deal. I told you this before. Cray Blitz was run as a client to the "driver". I told the driver the moves to make in a batch file, along with how long to wait after Cray Blitz started pondering before entering the next move. For the 16 cpu test, it might have pondered for 10 minutes before the opponent moved. This was adjusted for the 8, 4, 2 and 1 processor tests so that the ponder search got to the _same_ point. The only variable was the number of processors. Why is this so hard to understand? It wasn't hard for us to figure out how we wanted to run the test. It wouldn't be hard to do this with Crafty today although I would have to rewrite the front-end driver again since it was also lost... > >Nowhere in the article he mentions the permanent brain problem, >which bob probably didn't realize. The 1 processor run didn't >do a permanent brain but simply ran at position 1, then 2 ply >were made and it ran at position 2 etcetera. The position >followed a game and usually the move made also matched of >course the output. This means that after making 2 ply, >especially if you consider it was a near to fullwidth search >with many extensions, could excellently use the hashtable >info from previous output. That is wrong. If the 16 cpu test pondered until producing a PV at depth=12, then the 1 cpu test was forced to ponder for that length of time necessary to reach the same depth. I mentioned this in the paper, and mentioned how much time this burned on a one cpu machine. Your concept is both wrong, and broken, as usual... > >With 4 processors of course that helped even more to get a >better speedup than with cleaned hashtables. We can forgive >bob doing the test like this, because the 16 processor >output was obviously expensive system time considering >the machine has very little processors with very fast >shared memory. > >However the 16 cpu output had a problem the 1-8 cpu's >didn't have. > >if diep has a mispredicted move and searches for say a minute or >3 at the wrong position, then the hashtable is completely loaded >with the wrong information. So the speedup is as if the hashtable >is cleared then. Sometimes even slightly negative, sometimes >slightly positive. That might be the case with _your_ implementation, but it isn't a problem with mine. First, I can tell that the current table entries don't come from the current search which makes replacement easy. Second, even if the opponent makes a different move, many times it is a transposition of what is in the table and the table values are _still_ good. Don't know why you can't do this stuff right, but I can... > >We can see from the search times and such that Cray Blitz DID have >a permanent brain, some moves are considerably faster played than >others. Correct. It was pondering in all tests, from 1 processor to 16. > >Mchesspro played a lot of times not predicted moves. We can see that >too from the search times. Not a lot, but some, yes... > >Obviously the 8 processor rerun had a huge advantage compared to the >16 processor output, but didn't have that advantage to 4 processors. I _love_ your hand-waving. You are doing it again. Explaining why something can't possibly work, without even knowing how it worked or how the test was run. Rave on, of course... > >So Bob *had* to fake the outputs of 1-8 processors or his 16 processor >thing would look silly though it wasn't at all. Another "proof"??? > >>in computerchess it only goes about the times. nothing >>else matters. not the 'speedup' number, but the >>times. if you invent yourself a speedup number and >>calculate based upon that the time, then your whole >>thing is a big lie simply. >> >>It is provable that all search times from 1-8 cpu's >>at all tests are completely not true. they are about >>a factor 2 too fast in order to let the 16 processor >>look good. >> >>We do not talk about round off errors here. But a completely >>faked 1-8 cpu's time picture. >> >>On September 03, 2002 at 21:02:03, martin fierz wrote: >> >>>On September 03, 2002 at 20:32:00, Vincent Diepeveen wrote: >>> >>>>On September 03, 2002 at 18:49:08, martin fierz wrote: >>>> >>>>the problem is the speedups he didn't round off. >>>>the problem is the search TIMES. No way to see his >>>>search numbers as rounded off numbers. please see the >>>>table: >>>> >>>>pos 1 2 4 8 16 >>>>1 2,830 1,415 832 435 311 >>>>2 2,849 1,424 791 438 274 >>>>3 3,274 1,637 884 467 239 >>>>4 2,308 1,154 591 349 208 >>>>5 1,584 792 440 243 178 >>>>6 4,294 2,147 1,160 670 452 >>>>7 1,888 993 524 273 187 >>>>8 7,275 3,637 1,966 1,039 680 >>>>9 3,940 1,970 1,094 635 398 >>>>10 2,431 1,215 639 333 187 >>>>11 3,062 1,531 827 425 247 >>>>12 2,518 1,325 662 364 219 >>>>13 2,131 1,121 560 313 192 >>>>14 1,871 935 534 296 191 >>>>15 2,648 1,324 715 378 243 >>>>16 2,347 1,235 601 321 182 >>>>17 4,884 2,872 1,878 1,085 814 >>>>18 646 358 222 124 84 >>>>19 2,983 1,491 785 426 226 >>>>20 7,473 3,736 1,916 1,083 530 >>>>21 3,626 1,813 906 489 237 >>>>22 2,560 1,347 691 412 264 >>>>23 2,039 1,019 536 323 206 >>>>24 2,563 1,281 657 337 178 >>>> >>>>That's not rounded off numbers at all. >>> >>>hi vincent, >>> >>>well, if i calculate time(1processor)/time(2processors) for this >>>table, one thing is obvious: all numbers i get are rounded to 0.1 >>>already. this means that the numbers given in the table are not >>>numbers he measured at all. if bob claimed that he *measured* these >>>numbers, yes, i would conclude (like you did i think) that his >>>data was fabricated (with 100% certainty). >>>but i think bob admitted that somewhere else in this thread? >>>something like that he measured speedups, rounded them, and just >>>calculated the times with the help of his rounded speedups? of >>>course, this is not what you would want to do, so the paper is >>>definitely flawed, but not invalid because of that flaw. >>> >>>anyway, if i then calculate the average speedup of 2 vs. 1 processor >>>it comes out as 1.96, which bob with his "newmath" would report as 2.0. >>>which is exactly what he did. >>> >>>aloha >>> martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.