Computer Chess Club Archives


Search

Terms

Messages

Subject: Multithreading under windows

Author: Vincent Diepeveen

Date: 06:58:39 12/05/00

Go up one level in this thread


On December 04, 2000 at 20:48:59, Robert Hyatt wrote:

>On December 04, 2000 at 15:53:36, Vincent Diepeveen wrote:
>
>>On December 04, 2000 at 15:46:14, Robert Hyatt wrote:
>>
>>>On December 04, 2000 at 15:33:51, Vincent Diepeveen wrote:
>>>
>>>>On December 04, 2000 at 15:25:34, Eugene Nalimov wrote:
>>>>
>>>>>On December 04, 2000 at 15:09:37, Vincent Diepeveen wrote:
>>>>>
>>>>>>On December 04, 2000 at 14:58:17, Andrew Dados wrote:
>>>>>>
>>>>>>>On December 04, 2000 at 14:53:02, Bo Persson wrote:
>>>>>>>
>>>>>>>>On December 04, 2000 at 14:43:06, Andrew Dados wrote:
>>>>>>>>[...]
>>>>>>>>>Dear Ernst.
>>>>>>>>>
>>>>>>>>>First I find your post (and many past ones) of very little merit, just 'flagrant
>>>>>>>>>commercial exhortations' at best. Second you can't ask everyone to read your
>>>>>>>>>book to understand what you mean if you can't write it clearly in few lines. And
>>>>>>>>>finally I fail to see what part of Vincents post was utter nonsense.
>>>>>>>>>
>>>>>>>>>-Andrew-
>>>>>>>>
>>>>>>>>Hey!
>>>>>>>>
>>>>>>>>The book is also his thesis. Maybe it is difficult to give a half page summary
>>>>>>>>of several year's work!
>>>>>>>
>>>>>>>Maybe... But giving same answer for each problem ('Go buy my book!') suggests
>>>>>>>something else to me.
>>>>>>>-Andrew-
>>>>>>
>>>>>>Let's not try to imagine what happens when he would have had a
>>>>>>commercial program.
>>>>>
>>>>>So you have a commercial program?
>>>>
>>>>What is 'he' referring to in the above sentence?
>>>>
>>>>>>Note that i used about 184mb RAM in 3 different caches with EGTB
>>>>>>information stored.
>>>>>>  - 150mb transpositionhashtable ==> i store EGTB there too in a special way
>>>>>
>>>>>So it is not exclusively EGTB cache, right?
>>>>
>>>>of course not. also transpositiontable but with some obvious
>>>>additions to let it perform better for egtb.
>>>>
>>>>>>  - 2x2mb egtb cache
>>>>>>  - 2x15 mb wasted for just the egtb indices as i'm dual processor.
>>>>>
>>>>>Trash. There is only one set of EGTB cache and indices, regardless of number of
>>>>>CPUs, at least in the code I wrote.
>>>>
>>>>I'm multiprocessor, please explain me how!
>>>>
>>>>I only saw some multiTHREADING code.
>>>>
>>>>There is a small but existing difference between the 2.
>>>
>>>
>>>Eugene doesn't know that you aren't using threads... you are creating
>>>new processes with nothing shared except what you explicitly share via
>>>shmget()/shmat().  I don't think that is the way to do this on an SMP
>>>machine, but that is your choice, of course.  And with the way you have
>>>implemented things, there _are_ 4 copies of the cache, since your processes
>>>don't share everything implicitly as is done using threads.
>>
>>>That is one of many disadvantages...
>>
>>First of all multiprocessing is faster as multithreading. I can tell you
>>some horror multithreading stories about windows if you want to.
>
>Horror stories, sure.  But faster?  hardly.  There is _no_ speed difference
>between threading and multiple processes, so long as you start the threads
>or processes once and then keep them around forever.  The advantage is easier
>sharing of data, open files, etc.  Eugene's code is _specifically_ written to
>work in a threaded SMP environment, with the appropriate locks to prevent bad
>things from happening.  In this case, threading is far superior, because you
>will be reading from the tablebases with _four_ processes.  No way the disk
>drives can keep up with that.  With threading, the processes can share the
>data and avoid 75% of the I/O at times...
>
>
>
>>
>>Secondly Eugene could have remembered as i several times posted it,
>>also emailed it to him asking how to fix it. That's been some time ago
>>i guess perhaps forgotten therefore.
>
>
>I hate to say it like this, but not using threads makes no sense to someone
>that has been programming for a long time.  Which includes Eugene _and_ myself.
>I had to stop and think to figure out why you have 4x the cache.  Because doing
>things that way is just so far beyond what I would consider doing...  I do
>threads on unix and windows and have _no_ problems of any kind...  And can't
>envision not using threads in chess...
>
>
>
>>
>>Sharing the indices using multiprocessing is indeed not very easy.
>
>
>It is _already_ done in his code.
>
>
>
>>
>>I see no disadvantages towards multiprocessing, as it saves me
>>from a pointer which you need but i don't need. For clusters
>>or running over the network i see advantages, but under linux for
>>sure it's a disadvantage as not only this problem needs to be
>>solved.
>
>The pointer can be done away with in an iterated rather than a recursive
>search.  Cray Blitz was iterative in FORTRAN which has _no_ pointers of any
>kind.  It still threaded just fine.  I chose to use the pointer approach for
>my own sanity...
>
>
>
>
>>
>>In windows multiprocessing sure is a major advantage over multithreading,
>>but i only start to realize that pretty recently after having huge
>>problems with threads and GUI.
>
>
>How is it an advantage?  You open everything 4 times, you have 4 copies of
>everything.  Sharing data has to be done explicitly.  I don't see any
>advantage at all, and I can see several disadvantages when you factor in the
>tablebase problem to name one.



>
>
>>
>>If you do something in the GUI which is doing graphical things,
>>then it seems windows is blocking all other threads of the process.
>
>That doesn't make any sense at all.  NT handles threads perfectly.  Otherwise
>the engine would hang when xboard did the animation stuff (winboard, actually)
>but we don't see this...

crafty is a console program.
Graphical things are a bit different under windows, i don't
talk about linux here, i have no opinion about multithreading
in linux at all, only know multiprocessing sucks in linux as
it doesn't have shared anonymous memory at all.

Windows is a very sequential system with regard to graphics.
If some key is hit or a mouse clicks whatever and it goes
to LRESULT CALLBACK WndProc() or a similar LRESULT CALLBACK
function then all other threads from the winmain process
get blocked. Note that this is logical also if you look at it afterwards
as windows by doing that prevents that you get parallel problems.

Suppose your timer and wndproc do things at the same time! All windows
applications might start to crash!

Of course other processes don't get blocked.

I feel this as a big disadvantage for a program that's realtime
searching and i didn't see any
great solution to this problem other than being happy i was
multiprocessing already.

So in short i'm both multithreading and multiprocessing at the
same time. multiprocessing for engine multithreading for the rest
of the interface and i'm real happy with it.

>
>
>
>
>>
>>>
>>>
>>>>
>>>>>Eugene
>>>>>
>>>>>>Note i probe also in the qsearch which most people don't do. I initially
>>>>>>didn't do it either but then started drawing some games on the internet
>>>>>>because i didn't do it.
>>>>>>
>>>>>>But for sure i don't want to waste another 16mb RAM from the expensive
>>>>>>RAM i have now for another EGTB cache.
>>>>>>
>>>>>>Instead more likely is that i convert the EGTB to my own format after
>>>>>>which users who usually don't have an additional 15mb for egtb indices
>>>>>>to let them do without.
>>>>>>
>>>>>>>>
>>>>>>>>I have read the book and found it very interesting, full of new information and
>>>>>>>>a very good buy.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>BoPersson
>>>>>>>>bop@malmo.mail.telia.com



This page took 0.01 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.