Author: Uri Blass
Date: 04:50:35 10/11/00
Go up one level in this thread
On October 11, 2000 at 07:08:53, Graham Laight wrote: >On October 10, 2000 at 14:50:20, Vincent Diepeveen wrote: > >>On October 10, 2000 at 10:07:03, Graham Laight wrote: >> >>>On October 10, 2000 at 08:36:10, Vincent Diepeveen wrote: >>> >>>>On October 10, 2000 at 07:05:45, Graham Laight wrote: >>>> >>>>>It seems to me that PCs' results against GMs are tapering off into a flat line. >>>>>The current style of program may have come as far as they can go. >>>>> >>>>>The battle to generate the highest NPS score is no longer improving the >>>>>computers' performance against humans. Even Deep Junior running on a quad >>>>>processor is only able to score 4.5/9 against the top players. >>>>> >>>>>With dozens of programmers competing to make the "final push" to get programs >>>>>ahead of humans, to impartial observers it looks like the harder they push, the >>>>>more the bandwagon gets stuck in the mud. >>>>> >>>>>Programmers also have to remove knowledge from their eval fns to score higher >>>>>against their computer opponents. >>>>> >>>>>Looks like a doubling of NPS no longer provides an extra 50 Elo rating against >>>>>humans - nothing even close, in fact. >>>>> >>>>>I wonder whether some of our common assumptions about how speed and knowledge >>>>>affect ELO ratings are wrong. >>>>> >>>>>I think we mostly assume that the return on both knowledge and speed look like >>>>>the picture below: >>>>> >>>>> | >>>>> | >>>>> | ** >>>>> | ************************** >>>>> | ** >>>>> ELO | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> |* >>>>> |* >>>>> |* >>>>> |--------------------------------------------------------------- >>>>> Either Speed or Knowledge >>>> >>>>remove knowledge from this axis. >>>>not many very knowledgeable programs so far have had a chance to >>>>compete in big tournaments or in big matches. >>> >>>Agreed - the shape of the graph is proven for neither knowledge nor speed. >> >>i would sure want to say that there is clear proof for the speed side. >>more knowledge as the average testset used to proof a new algorithm in >>JICCA. > >The diminishing returns of speed is known. What is not known is whether the >speed graph jumps and plateaus. I suspect that this effect is caused by >programmers changing the way their programs work as the speed increases. > >>>It just seems as if each time the top level plateaus, it stays at the plateau >>>longer than last time. And possibly jumps upwards less when it starts to move >>>again. >> >>though this looks to me a logical assumption the granularity definitely >>is so far defined by commercial challenges. obviously those challenges >>are done by companies who want their investments seen paid back, to their >>viewpoint is not scientific. > >I agree you can't trust 100% what commercial folk tell you. Even impartial SSDF >took a long time to admit that their list was inflated - but I'm glad that they >eventually did, and took appropriate corrective measures. > >>>>> >>>>>But what if, in reality, one or both of them actually looked like this? >>>>> >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | ** >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | ** >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | ** >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> | * >>>>> ELO | * >>>>> | * >>>>> | ** >>>>> | * >>>>> | * >>>>> |* >>>>> |* >>>>> |--------------------------------------------------------------- >>>>> Either Speed or Knowledge >>>> >>>>>In other words, shooting up, plateauing for a while, then shooting up again - >>>>>and so on. It's possible that, because chess programmers vary the amount of >>>>>expertise between 20 and (say) 500 distinct pieces of knowledge, they've found a >>>>>plateau (probably the 2nd one), and, angry about being beaten by someone with >>>>>less knowledge but higher NPS, have refused to go down the knowledge route >>>>>seriously. Also, from many years of reading postings in this group, it is >>>>>apparent that NPS, and techniques to raise it, is where the focus lies with this >>>>>particular group of people. >>>> >>>>obviously fritz3 would search 17 ply at nowadays hardware if it was >>>>updated a bit to 32 bits + 8 bits code and hashtable would get implemented >>>>a bit better. Yet fritz6 doesn't get that. >>> >>>I have to disagree with this. From my understanding: >>> >>>* Fritz 3 used root processing to achieve its speed. But root processing at 17 >>>ply would be completely useless - so lowering the NPS per MHz is inevitable. >> >>for the depth versus playing strength graph this is not relevant, >>idem for the arguments below. >> >>However for the more knowledge + deeper search versus performed rating >>i definitely think the fact that all kind of programs get rewritten is >>showing a clear thing. So far it's not clear to what junior gets rewritten >>to, but as soon as Amir wants to talk about that, we might have more >>datapoints. > >Amir has previously said at CCC that he can implement extra knowledge almost for >free (in terms of processor clock cycles) - which begs the question, "Why isn't >Junior the best chess player in the world?". I guess that one of the reasons is that Amir cannot implement everything that he wants for free. There are things that he can implement for free but there are things that he cannot. When I was a beta tester of Junior Amir replied about part of my ideas that he cannot do it. I do not know much about Junior but I know that if the situation today is similiar to the situation one year ago then there are important things that Junior cannot evaluate(It does not mean that it is a piece square table program or something close to it like someone said in the past). I also know that Junior6a has problems with stalemate detection in some rare positions(It says mate against itself when there is a very simple stalemate combination). I know that this problem was not fixed in Junior6a inspite of the fact that I told Amir about it. If it is easy for Amir to add this knowledge to the evaluation without cost then I guess that he could add this knowledge. The only reason that he did not do it may be that it is not easy to do it or that the cost in speed is significant. Uri
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.