Author: Joseph Ciarrochi
Date: 22:02:00 01/18/06
Go up one level in this thread
> >Hi Joseph, > >I don't think it would provide any meaningful data to be honest. I'd have >thought that the main purpose of engine v engine testing was to determine which >performs better given equal conditions. >Just my opinion, > >Regards, Graham. Yes, I would not want to replace engine versus engine tournments. I too love to see these. And to be honest, i don't think ceiling or floor effects are going to make a huge practical difference (though this is an empirical question) I'll use an example from another domain to make my suggestion. Let's say you are evaluating the items on an IQ test. You can do an item analysis to establish item sensitivity. Some items will discriminate very well between people above and below average in IQ, but will not discriminate well between two people high in IQ. Other items will discriminate between people with a 120 iq verus those with a 140 iq, but will not discriminate between below and above average IQ people. Now, if I have limited time, and I know i am trying to discriminate between two high IQ people, then I will use the best subset of items for this purpose. Returning to the world of chess, let's say you make minor modifications to your engine and you want to determine if it is statistically better. I would say you've got to find the engines that will be most sensitive to these differences. So you don't want your oponents to be too strong or too weak. Otherwise, maybe differences will be masked (and i say maybe, because we can't say their is a ceiling or floor without analyzing the data. I can do this with what is posted on the cegt....but i am suppose to be working on my real job now :( ) Anyway, I think it all depends on your purpose. best Joseph
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.