Author: Don Dailey
Date: 13:58:07 01/03/98
Go up one level in this thread
>I think the risk you take is a sort of "inbreeding". One example: > >Crafty is a "null-move" searcher. At shallow depths, null-move can hide >simple threats quite easily. I used to see this when playing on ICC >using >a P5/133 machine. Once I moved to the P6/200 and tuned things better >for >that machine, skill went *down* on the P5. > >If I play crafty vs crafty on the P5, I don't see the threat problem >because >*neither* one sees the threats since their searches are similar (I am >assuming >I am working on eval changes, but it could also be search changes as >well). >As a result, results at short and long time controls will be the same. >But >if I play another program (like Genius for example), at short time >controls >Crafty will suffer more than at longer time controls, because the longer >time controls tend to hide the null-move "weakness" that shallow >searches >expose quickly. > >So it is possible to draw conclusions from short games that are wrong, >if >the short games are A vs A, rather than A vs B... The "inbreeding" you talk about is something everyone believes and everyone talks about. I believe it too and yet have never seen it! As I mentioned I periodically look for this and only find it at very short levels, less than 5 ply. And even this is very occasionally. I can almost predict when it will happen, it always involves interesting tactical extensions or algorithms designed to pick up tactics. When I test against other chess programs, I have not seen the inbreeding affect either although it makes sense to me that it should exist. If I do something that makes my program stronger, it always performs better against other programs too. I think intransitivity comes into play where you address a weakness in your program that is not a problem against program A but a big problem against program B. After making this change you scratch your head because your program now does much better against program B but about the same or slightly worse against program B. If the issue, as an example, is king safety then perhaps your program had no problems against program A (who also had weak king safety) but you were getting crushed constantly by program B's king safety! But when you address a problem like this, your test results will get better with self testing because now your program not only knows how to defend, but automatically knows how to attack. In my case I find that if it helps in self play it will help against other opponents although perhaps some more than others. So in summary I definitely believe you can tune your program to play one opponent better but this invariably makes your program stronger anyway. I think intransitivity among programs is fairly weak and does not prevent you from self-testing effectively. For the same reason it should not prevent you from autotesting either. Think of self testing as just another opponent, not worse or better to test against. The REAL benefit from testing lots of opponents is that each opponent may be able to expose weaknesses the others cannot. About this: >Crafty is a "null-move" searcher. At shallow depths, null-move can hide >simple threats quite easily. I used to see this when playing on ICC >using >a P5/133 machine. Once I moved to the P6/200 and tuned things better >for >that machine, skill went *down* on the P5. Are you tuning for the machine or the speed of the machine? It sounds like you have something non-optimal in there somewhere. If you know what you did you must be able to make the thing self adjust so it will play close to optimal at any time control? -- Don
This page took 0.01 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.