Author: Robert Hyatt
Date: 07:10:39 12/24/98
Go up one level in this thread
On December 23, 1998 at 20:36:30, Roberto Waldteufel wrote: >Hi all, > >I would like to share an idea I am currently looking into concerning hard disc >access. For many chess programs that use tablebases hard disc access can be a >serious problem when there are many hits on the tablebases deep in the search. >If the disc accesses could be sped up, the program would benefit very much in >these types of positions. > >The idea is this: bypass everything and go directly to the data via hardware. Of >course this is technically much more complicated than simply opening the file >and retrieving the required byte in the normal way, but I suspect the speed >improvement may be substantial if it can be done, which I think it can. The >fastest way to access the hard disc is via the hardware ports. We need to know >the sector and the sector offset for the data in the file, but this information >can be read from the FAT. By using ports, we load just the data we want from >just the sector we want (not a cluster of sectors), and cut out all the >operating system overhead. The point is that we know the size of the file, we >know it is not going to be altered while the program runs, so all the FAT >information should be readable at start-up time, and then instead of calculating >a simple byte location in the file for a given position, we calculate the actual >sector/offset on disc and read the data via the ports directly from disc. > >There is a problem in that Windows disallows direct hardware access, and goes to >great lengths to conceal the necessary API call (CallVxD0) in an effort to >prevent programmers from doing precisely the sort of thing I am proposing. By >writing the port access code in a virtual device driver, we can obtain the >required Ring0 privelege to read/write not only ports, but any area of memory >(protected or not). Micro$oft removed the vxdcall function from the export list >for kernel32.exe when the function's potential uses were publicised, but it is >still present - it just takes a bit of hacking to get at it, but there is a >"back door": it is still there and very useful indeed! Using this API call, I >have now a VxD in place that can, among other things, read and write the >hardware ports under 32-bit windows I think, if I can make this work with disc >accessing, the potential could extend beyond chess programming to other disc >byte-data intensive computer operations. If anyone has already tried anything >along these lines or has any comments I would be intersted to hear from you. > >Merry Christmas to all, >Roberto If you try this, you'll find it doesn't help a bit. The problem is not the operating system overhead, it is the rotational latency and seek delays that are still *very* slow compared to the speed of the processor. IE I am now running the fastest disks I know about, IBM drives running at 10,000 rpm, with an average total seek time of 5ms. 5ms means you can do about 200 i/o operations every second assuming a seek between each pair of reads (which is pretty normal in a tree search). Nothing you can do will reduce this overhead, because it is in the device, not in the operating system. There are other ways to keep the nps from dropping too low. IE in Crafty, I now have an adaptive probe depth algorithm that prevents my probing so much that my nps drops to 2-5000 nps, for example. I rarely drop below 200K now and only for a move or two before it adapts and perks the speed back up to near-normal...
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.