Author: Vincent Diepeveen
Date: 05:36:10 10/10/00
Go up one level in this thread
On October 10, 2000 at 07:05:45, Graham Laight wrote: >It seems to me that PCs' results against GMs are tapering off into a flat line. >The current style of program may have come as far as they can go. > >The battle to generate the highest NPS score is no longer improving the >computers' performance against humans. Even Deep Junior running on a quad >processor is only able to score 4.5/9 against the top players. > >With dozens of programmers competing to make the "final push" to get programs >ahead of humans, to impartial observers it looks like the harder they push, the >more the bandwagon gets stuck in the mud. > >Programmers also have to remove knowledge from their eval fns to score higher >against their computer opponents. > >Looks like a doubling of NPS no longer provides an extra 50 Elo rating against >humans - nothing even close, in fact. > >I wonder whether some of our common assumptions about how speed and knowledge >affect ELO ratings are wrong. > >I think we mostly assume that the return on both knowledge and speed look like >the picture below: > > | > | > | ** > | ************************** > | ** > ELO | * > | * > | * > | * > | * > | * > | * > |* > |* > |* > |--------------------------------------------------------------- > Either Speed or Knowledge remove knowledge from this axis. not many very knowledgeable programs so far have had a chance to compete in big tournaments or in big matches. > >But what if, in reality, one or both of them actually looked like this? > > | * > | * > | * > | * > | ** > | * > | * > | * > | * > | * > | ** > | * > | * > | * > | * > | * > | * > | ** > | * > | * > | * > | * > | * > ELO | * > | * > | ** > | * > | * > |* > |* > |--------------------------------------------------------------- > Either Speed or Knowledge >In other words, shooting up, plateauing for a while, then shooting up again - >and so on. It's possible that, because chess programmers vary the amount of >expertise between 20 and (say) 500 distinct pieces of knowledge, they've found a >plateau (probably the 2nd one), and, angry about being beaten by someone with >less knowledge but higher NPS, have refused to go down the knowledge route >seriously. Also, from many years of reading postings in this group, it is >apparent that NPS, and techniques to raise it, is where the focus lies with this >particular group of people. obviously fritz3 would search 17 ply at nowadays hardware if it was updated a bit to 32 bits + 8 bits code and hashtable would get implemented a bit better. Yet fritz6 doesn't get that. So obviously search depth is not the holy thing to achieve. Now that in most games the programs search >= 10 ply it's obvious that better knowledge comes first. Less and less commercial programs are pruning using dubious methods like FHR, evaluations get less lazy, and less moves get pruned on alpha in the qsearches. It's all obvious that a more accurate score is preferred nowadays above tactical search depth. Yet the big impact was of course the nullmove, which with the introduction of bigger processors kicks butt bigtime. Yet at the same time as starting to use nullmove i notice in diep (don't know whether other notice that too) that if it's used with last ply pruning in one or another way, that there is a clear loss of knowledge, as it takes at least another few ply to get the same move as best move just being seen as 0.004 better sometimes as the previous best move which in fact is not that a good move compared to the newly selected move. Also hashtables influence a lot. Even small search depths like up to 9 ply, my diep needs 2.5 times more nodes WITHOUT transposition + best move storage in obvious to sort positions! >Could it be that, for all these years, we've allowed ourselves to be fooled by >the apparent "computer friendliness" of chess, followed the same path >progressively, and neglected to address the problems with knowledge coding, >knowledge selection (during the game), and knowledge acquisition - which are >themselves very interesting computing problems? >Clinching evidence: humans are still able to beat computers. They aren't using >NPS, therefore they must be using knowledge. Could have been my conclusion, but i would never manage to write it down that polite! >-g
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.