Author: Andrew Wagner
Date: 20:09:16 05/25/04
Go up one level in this thread
On May 25, 2004 at 22:37:58, Will Singleton wrote:
>On May 25, 2004 at 22:01:31, Andrew Wagner wrote:
>
>>On May 25, 2004 at 21:00:52, Will Singleton wrote:
>>
>>>On May 25, 2004 at 20:09:57, Andrew Wagner wrote:
>>>
>>>>On May 25, 2004 at 19:36:04, Will Singleton wrote:
>>>>
>>>>>On May 25, 2004 at 17:44:55, Andrew Wagner wrote:
>>>>>
>>>>>>I do a lot of reading through CCC archives. I use the search engine from here,
>>>>>>and also I'm in the process of reading through the old archives systematically
>>>>>>using the offline reader (I'm in the fall of 2001 currently, I think). Anyway,
>>>>>>sometimes I run across a nugget that makes me just stop and go "whoah". Here's a
>>>>>>quote from one of Bob's posts, originally about hashing algorithms:
>>>>>>
>>>>>>>I think the key to improving a program, once it plays legally, is to develop
>>>>>>>a methodology to carefully profile the code, find the hot spots, and then find
>>>>>>>ways to speed up those hot spots. But all the while paying _careful_ attention
>>>>>>>to the overall node counts on a wide range of test positions. A 1% speedup is
>>>>>>>of no use at all if you introduce an error that happens once every billion
>>>>>>>nodes. I can search that many nodes in 15 minutes. I can't stand errors that
>>>>>>>frequently. I have what would probably be called a "zero-tolerance for errors"
>>>>>>>in Crafty. If I make a change that should only make it faster or slower, then
>>>>>>>the node counts must remain constant. If they don't I debug until I find out >why and fix it.
>>>>>>
>>>>>>This is a fantastic point. Maybe somewhat obvious to our more experienced
>>>>>>members, but certainly words of wisdom for us newbies. So, my question is, what
>>>>>>methods are you all using for profiling your code? How do you go about
>>>>>>identifying and fixing your hotspots? Do you have a particular test suite you
>>>>>>use, or what? Andrew
>>>>>
>>>>>I'm surprised Bob would say that profiling is important so soon in the
>>>>>development process; perhaps there's some missing context. Profiling is, imho,
>>>>>about the last thing you'd want to do.
>>>>
>>>>Here's the link, so you can read it in context, if you'd like:
>>>>http://chessprogramming.org/cccsearch/ccc.php?art_id=112972
>>>>
>>>>>
>>>>>1. Fix bugs in movegen, using perft tests.
>>>>
>>>>He does say "once it plays legally". To me, that implies a bugfree movegen...
>>>>
>>>>>2. Write a very simple, bug-free eval.
>>>>>3. Concentrate on move-ordering, which is crucial to making the tree small.
>>>>>Develop methods for measuring the quality of your ordering, don't only look at
>>>>>node counts.
>>>>>
>>>>
>>>>I would count these as what he calls "hot spots". Especially move ordering
>>>>(though good eval helps move ordering).
>>>>
>>>>>Don't spend a lot of time on arcane or new ideas until you're certain what you
>>>>>have is bug-free. Especially make sure your transposition code is simple and
>>>>>effective, tons of problems result from bad hashing.
>>>>
>>>>I would also be interested in a process for this. What process do you use to
>>>>really be absolutely sure your program is bug free? Especially your hash table
>>>>code. E.g. at the moment, I think Trueno is bug free, more or less. But I
>>>>haven't found a good thorough test to give it that will tell me.
>>>>
>>>>>
>>>>>Once you have a good, stable platform to build on, you can be sure that your
>>>>>future experimentation will be productive.
>>>>
>>>>But what does the "build on" process consist of? That's the question. Bob's
>>>>answer is this profile/find hotspots/fix-while-watching-node-counts process. But
>>>>how do you implement this?
>>>
>>>It's a good area for discussion. By way of background, a question: there are a
>>>few starter programs which seem to be debugged and simple, like gerbil and tscp.
>>> How does Trueno do against those?
>>
>>Trueno usually gives TSCP a run for its money, but I don't think it's managed
>>better than a draw yet. I haven't tried it with gerbil...I can't seem to get
>>gerbil working with winboard. I'll have to fool with that some more.
>
>I just need to be clear, do you mean Trueno usually gets 50% or less in a match
>against tscp, or do you mean Trueno hasn't yet beaten tscp in a game?
Ooooh, you're gonna make me say it aren't you? Ok, ok, as far as I can remember
Trueno has not yet EVER beaten TSCP in a game. Now, to soothe my pride, let me
add that I got Gerbil working, and Trueno just won this interesting little 30 0
game against it:
[Event "30 0 game"]
[Site "?"]
[Date "2004.05.25"]
[Round "?"]
[White "Trueno"]
[Black "Gerbil"]
[Result "1-0"]
[Opening "King's Indian: Andersson variation"]
[ECO "E92"]
[NIC "KI.18"]
[Time "23:01:01"]
1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 O-O 6. Be2 e5 7. dxe5 dxe5 8.
Be3 Qxd1+ 9. Rxd1 Bg4 10. h3 Bxf3 11. Bxf3 Rc8 12. h4 c5 13. h5 gxh5 14. Nd5
Nxd5 15. cxd5 h4 16. Bg4 Re8 17. Bxc5 Na6 18. Ba3 Bf6 19. Rd3 Red8 20. Rf3
Kg7 21. Rb3 b6 22. Rc3 b5 23. Rb3 Rab8 24. Be2 Rdc8 25. Rc3 b4 26. Rxc8 Rxc8
27. Bxa6 Rc1+ 28. Ke2 Rxh1 29. Bxb4 h3 30. gxh3 Rxh3 31. Bc5 Rh4 32. Bd3 a5
33. Bb6 Kf8 34. Bxa5 Ke7 35. a4 h5 36. Bb4+ Kd7 37. Bc5 Kd8 38. b4 Rh2 39.
d6 Rh4 40. a5 Kc8 41. a6 Bg5 42. d7+ Kc7 43. Bd6+ Kc6 44. b5+ Kxd7 45. a7
Kxd6 46. a8=Q Rg4 47. f3 Rg2+ 48. Kf1 Rd2 49. Qd5+ Kc7 50. Qxf7+ Kb6 51.
Qg6+ Ka5 52. Qxg5 Rxd3 53. Ke2 Rb3 54. Qxe5 Rxb5 55. Qa1+ Kb6 56. f4 Ra5 57.
Qc3 Rc5 58. Qb2+ Rb5 59. Qd4+ Ka5 60. Qh8 Ka4 61. f5 Rb4 62. Kd3 Rb3+ 63.
Kc4 Rb4+ 64. Kd5 Rb5+ 65. Kc6 Rb1 66. Qxh5 Kb4 67. f6 Rc1+ 68. Kd6 Rc8 69.
e5 Rd8+ 70. Ke7 Rb8 71. Qh1 Kc5 72. f7 {Black resigns} 1-0
I should also point out that I have not done a lot of testing with these
engines, because my arena doesn't seem to be able to run tournaments correctly.
If it did, I'd be running a lot more of them, for testing purposes. As it is,
all the tests I do get done by hand, one game at a time, with winboard.
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.