Computer Chess Club Archives


Search

Terms

Messages

Subject: Re: Puzzled about testsuites

Author: José Carlos

Date: 01:23:04 03/11/04

Go up one level in this thread


On March 11, 2004 at 04:04:01, Uri Blass wrote:

>On March 11, 2004 at 03:08:36, José Carlos wrote:
>
>>On March 10, 2004 at 18:29:31, Uri Blass wrote:
>>
>>>On March 10, 2004 at 16:04:32, Dann Corbit wrote:
>>>
>>>>On March 10, 2004 at 14:41:47, Uri Blass wrote:
>>>>
>>>>>On March 10, 2004 at 14:23:29, Dave Gomboc wrote:
>>>>>
>>>>>>On March 09, 2004 at 16:05:15, Gian-Carlo Pascutto wrote:
>>>>>>
>>>>>>>Yet no top program does this, and they had a human correct it
>>>>>>>afterwards in Deep Blue. The conclusion should be obvious.
>>>>>>
>>>>>>Is that so?
>>>>>>
>>>>>>>If you can develop a *top level* evaluation function, better than
>>>>>>>good human tuning, solely on learning from a GM games database,
>>>>>>>you deserve an award. Nobody has succeeded before.
>>>>>>
>>>>>>Jonathan Schaeffer learned weights in Checkers [Chinook] without even using a
>>>>>>human games database (he used TD learning).  The weights he tuned score 50%
>>>>>>against his hand-tuned code.
>>>>>>
>>>>>>I learned weights in Chess [Crafty] using 32k positions, hill-climbing an
>>>>>>ordinal correlation measure.  It too scores 50% against the hand-tuned code.
>>>>>
>>>>>How many games and what time control?
>>>>>There is a difference if you score 50% with 2 games and with 2000 games?
>>>>>
>>>>>It is also possible that you get 50% against Crafty but less against other
>>>>>opponents.
>>>>>
>>>>>
>>>>>>Given Deep Sjeng's source code, I could zero its evaluation function weights,
>>>>>>and learn them from GM games to score 50% against the weights you have right now
>>>>>>too.
>>>>>
>>>>>You may be right but you cannot know about source code that you do not know.
>>>>
>>>>With his method, he will eventually reach a good result with any engine.
>>>>It uses generations, and discards the weaker ones absorbing the stronger ones.
>>>>After long enough waiting, it must become stronger.
>>>
>>>The question is how much time is long enough.
>>>It is clear that there is a way to find the best setting after enough time.
>>>
>>>Even the simple way of testing every possible setting can find the best setting
>>>if you only have 10^1000 years to wait.
>>
>>
>>  The number of possible settings is infinite, so this method can't be sure to
>>find the best settings.
>>
>>  José C.
>
>No
>
>The number of possible setting is finite but very big and if you have 200
>parameters and everyone of them can get 10 values you have 10^200 setting.
>
>I admit that practically you probably have thousands of numbers when most of
>them can get more than 10 values so you have near 10^10000 possible setting so
>you nead probably at least 10^10000 years to find the best setting by testing
>all of them and 10^1000 is not enough.
>
>It does not change my point in this discussion that the question if you find the
>best setting if you wait enough time is not important and in this case waiting
>until programs solve chess may be shorter wait.
>
>Uri

  Uri, the number is infinite because parameter values are integer and integer
numbers is a set with infinite elements.
  Besides, the "given enough time" concept implies a convergency, and thus an
iteratively better results (not necessary increasing accuracy all the time but
increasing as a tendency). Your "try all settings" has no convergency, that's
the difference.

  José C.



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.