Author: Scott Gasch
Date: 13:16:00 12/10/01
Go up one level in this thread
On December 09, 2001 at 21:09:17, martin fierz wrote: .... >i would have expected these two approaches to be equivalent, but the new one >drops a few % of speed overall. that's a lot, because the whole hashing stuff >only takes up a few % of my work, so it looks like that part is something like >half as fast as earlier. When you have a global array in your code it will live in the .data section of your binary image. When you allocate a big chunk of memory for your hashtable it will come from the process' heap. So the memory layout of your image before and after this change is dramatically different. The slowdown you are seeing is, I think, more likely a result of the changed memory layout than a penalty for accessing the hashtable memory the new way. The new way might result in one or two new instructions per hit, but that doesn't explain the kind of slowdown you are talking about. I saw the same thing happen when I modified some data structs in my chess engine to support multithreaded search recently. I saw an instant 15%+ decrease in engine speed. This was due to memory layout and was corrected, believe it or not, by rearranging the order of locals in a single function. My first bit of advice to you is to profile your code before and after the change. Be very suspicious of any routine that is taking longer... especially if it has nothing to do with the change. >another question i have is about malloc - under windows you have different >memory allocation functions like VirtualAlloc and HeapAlloc. HeapAlloc will not >allow allocations > 256MB, at least not on win98. i usually use malloc in C, >because i do not want to break the portability of my code. my windows book says>i should use VirtualAlloc for "large objects" (= more than 1MB), and HeapAlloc >for smaller objects. is malloc in C (MSVC 6.0) translated into a HeapAlloc? >could this be the reason that the program is slow, should i be using >VirtualAlloc? Malloc is HeapAlloc, at least in Win2k and WinXp. There is no maximum size you can HeapAlloc... this is limited by the size to which the process' default heap can grow... which is limited by the amount of non reserved virtual memory on your system. I don't know anything about Win9x though. I don't know why you would not want to allocate large memory chunks from the heap... as your book advises. It seems to me like if you have the memory on the machine it should not matter where it comes from. That said, you might try increasing the heap reserve / heap commit of your EXE with linker flags if you intend to allocate a huge hash table from its heap. This will at least make the allocation time faster. The other thing that may be slowing you down is that you are somehow swapping. Are you sure that the size of the hashtable you have allocated will fit in physical memory? I'd run perfmon and watch the "number of page faults per second" counter for your process while it runs. If this is not zero, expect to pay a large speed penalty. Personally, I use VirtualAlloc instead of HeapAlloc because: 1. On PARANOID builds I change memory protection on hash entries on the fly. Most of the time they are PAGE_READONLY so that if I have a bug that tries to touch hash memory when it shouldn't, I will AV. I change the page of an entry to PAGE_READWRITE just before I change it... and back to READONLY again when I'm done. I've never found a bug this way but someday I will. :) 2. On all builds I lock the hash table into physical memory. I do this by increasing the process working set and using VirtualLock. I think both of these things are only possible on memory buffers allocated via VirtualAlloc. But I'm not sure about either... All of this stuff is Win32 only so it makes your code less portable. But I've been careful to do this stuff in one module (system.c) and write UNIX equivalents where possible (you can do the same stuff on FreeBSD at least with memlock, memprotect, and mmap). Good luck, Scott > >mahalo > martin
This page took 0 seconds to execute
Last modified: Thu, 15 Apr 21 08:11:13 -0700
Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.