Author: Les Fernandez
Date: 10:33:27 06/10/03
Go up one level in this thread
On June 10, 2003 at 13:17:38, Robert Hyatt wrote: >On June 10, 2003 at 12:22:44, Les Fernandez wrote: > >>On June 10, 2003 at 11:12:28, Robert Hyatt wrote: >> >>>On June 10, 2003 at 03:34:27, Les Fernandez wrote: >>> >>>>On June 09, 2003 at 21:46:43, Dann Corbit wrote: >>>> >>>>>On June 06, 2003 at 11:27:27, Les Fernandez wrote: >>>>> >>>>>>I have an experiment I would like to run using the entire set of WAC positions. >>>>>>Can someone send me the epd strings for the entire set and then I was wondering >>>>>>if when I am done with them if someone would be interested in running the entire >>>>>>set. I think we may find some interesting results when I am done with them. >>>>> >>>>>It's been done by many people, including myself >>>> >>>>Hi Dann, >>>>I am aware that alot of work has been done with them but I am not aware of all >>>>the tpyes of things that have been tried. >>>> >>>>> >>>>>Are you sure you want to reproduce that information again? >>>> >>>>Hmmm when you say reproduce it again Dann what exactly has been done? >>>> >>>>> >>>>>What time control are you interested in? >>>> >>>>I was going to ask you about the time setting Dann. Perhaps if you can provide >>>>me with some data that has already been generated that might help me decide on a >>>>time. Basically I would be interested to see the right number of solutions >>>>(from the 300 set) with respect to time. How is it done? Does one set a total >>>>time and then see how many of the 300 positions his engine can solve or is it >>>>based on a constant set time per position? Also Dann in your opinion which >>>>engine seems to score well on the 300 positions and what kind of hardware was it >>>>running on? >>>> >>>>Although I will have to give it some thought I am not necessarily interested in >>>>the best engine. I am more interested I think in finding a time somewhere in >>>>between the best and the worst case scenarios. Will have to think about that. >>>> >>>>Thanks, >>>> >>>>Les >>> >>> >>>I have a test run from an older version, 60 seconds per move, that got 300 >>>correct on a 21264 / 600mhz alpha. Recent versions will not solve #230, >>>but get the rest in under 60 seconds on my dual xeon 2.8. Here is a run >>>at 10 seconds per move on my dual 2.8 xeon: >> >>Hi Bob, >> >>First thx for the stats. Let me ask you for your opinion about this test set. >>I mentioned the WAC set since I know many people use it as a gauge to see how >>their engines are performing. When you mention that your engine solved 299 of >>300 positions at 60 second time interval does that mean that each position was >>subjected to 60 seconds or did some only use 30 seconds and some longer and then >>you took an average or did you just give all positions 60 seconds. > >When I run such test sets, I give a time limit per position. IE in the >stuff I posted below, the time limit was 10 seconds per position. If one >is solved to a mate in 1 second, the search will terminate when that iteration >is done, and the remainder of the 10 seconds is "lost"... > >> What I would >>like to do is see if the set that I create gets to some of the solutions faster >>or slower. Should I be working with the WAC set Bob? > >Personally, I think it is too easy with today's hardware speeds. If I can get >299/300 (and I miss the last one because I don't value the resulting pawn >structure near high enough to make the rook sac look good) at 10 seconds per >move, then I don't think it is very useful. I would prefer a set of 300 >positions where I get say 100 right out of 300 at 10 seconds per move, as that >gives me lots of stuff to look at. WAC has easy positions, positions with many >correct (winning) moves, etc. Exactly Bob that would be an ideal set to work with. Uri just posted basically saying the same thing and I asked him for any idea as to what other sets exist that may fit more closely to the model that you mention here. Bob tihs would be a much better set to work with. Thanks, Les > > >> I mean when you ran this >>set you probably set it at 60 seconds and once the test was over reviewed which >>ones found the right answers. Maybe what I need to work with is the pgn data >>that was generated when you ran your test since that would show me, with a >>timeline, just what was found and how long it took. What are your thoughts Bob? >> >>Thanks, >> >>Les >> > >The little summary I posted was produced by the Crafty "test" command. It >will take epd positions, run them for whatever time limit you specify, and >count how many are right/wrong and print this when the test is finished. > > > > > >>> >>> >>>total positions searched.......... 300 >>>number right...................... 299 >>>number wrong...................... 1 >>>percentage right.................. 99 >>>percentage wrong.................. 0 >>>total nodes searched.............. 201494040 >>>average search depth.............. 4.6 >>>nodes per second.................. 2327527
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.