Author: Christophe Theron
Date: 11:51:05 12/02/01
Go up one level in this thread
On December 02, 2001 at 04:49:30, Harald Faber wrote:
>On December 02, 2001 at 01:48:03, Uri Blass wrote:
>
>>>It is common knowledge of programmers that 12Kb of hash tables is inferior to
>>>192Mb.
>>>
>>>So if the tester wants to use 12Kb of hash table, should the program:
>>>1) use 12Kb?
>>>2) refuse to run?
>>>3) use 192Mb no matter what?
>>
>>It is a different case
>>no user is using 12 kbytes but many users use the incomplete tablebases.
>>
>>It is the responsibilities of the testers to give the programs equal conditions
>>that mean to give them the same tablebases.
>
>Be sure that is what we had done. BUT: the TBs had been CONSISTENT TBs and not
>such a nonsense like endgame-turbo...
>
>>It is not the duty of the testers to use the full 5 piece tablebases because it
>>is possible that there is not enough place in the harddisk for them.
>>
>>Uri
>
>In times like these where 40GB cost $100-$150 (for more than a year now) I am
>sure many computer chess maniacs have enough space on their HD to install the
>complete 5-man TBs. I guess it is more the "how to get them cheaply?"
>However, you are right, it is not on us testers to find what happens if
>incomplete TB setup is used. Ido not have the "endgame turrbo" e.g., only the
>complete TBs. So what should I do? Move some of the 5-mans? Which ones? And then
>test again? No, this is nonsense. And others won't do that too.
>
>I repeat my question, considering that the complete 5-man TBs are available for
>free download or for little money:
>Should the programmer implement superfluous code although the TBs solve e.g. the
>"problem" mating with bishop+knight?
>Honestly, I would not waste my time or code too implement such a nonsense.
To be honest Harald, I have a different opinion, and you are going to understand
why by reading the following:
I believe that programs should not rely on TBs to solve KBNK. That's why Chess
Tiger has some internal logic to solve this case by itself because this program
is supposed to be able to work properly on a Palm, with no tablebases.
So I don't remove endgame knowledge in Tiger. I consider that Tiger does not
have to use tablebases and that it must be able to do without them. And if it
has them, it will use them and play even better.
I also believe that programs should take care of missing TBs and take
appropriate actions. That's something I'm going to implement in the next
evolution of Tiger.
However I also believe that a fair test about playing strength should not be
turned into a worst-case test or survival test.
I admit that Tiger is missing some code to take care about missing tablebases,
but under the correct setup (which is not difficult to achieve) this problem has
no impact on the measured playing strength.
So if the setup is incorrect and this problem affects the results, then the test
is not about real playing strength anymore.
For all the programs tested by the SSDF there are some minimal requirements.
There are parameters to set up correctly for every program and nobody denies
that.
Imagine that a program has, by default, the "permanent brain" turned OFF. What
is a fair test then? Is it to run the program as it is configured when it is
installed (so with PB off)? Or should the tester activate the "permanent brain"
checkbox before the match is started?
It is my understanding that the SSDF tries to do its best to get fair conditions
for the programs in order to measure real playing strength, not the playing
strength under hostile conditions.
That's why they ask the developpers what the best settings for their programs
are.
It is also my understanding that the TB problem has affected only a few games
played by Chess Tiger, so it would not be very hard for the SSDF to replay these
games with the correct settings.
Christophe
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.