Computer Chess Club Archives


Search

Terms

Messages

Subject: one more thing on cc-NUMA and linux

Author: Vincent Diepeveen

Date: 17:31:55 09/03/03

Go up one level in this thread


On September 03, 2003 at 20:20:05, Vincent Diepeveen wrote:

>On September 03, 2003 at 16:28:18, Robert Hyatt wrote:
>
>>On September 03, 2003 at 15:27:15, Vincent Diepeveen wrote:
>>
>>>On September 03, 2003 at 13:15:48, Robert Hyatt wrote:
>>>
>>>>On September 03, 2003 at 12:23:08, Vincent Diepeveen wrote:
>>>>
>>>>>On September 03, 2003 at 10:54:37, Sune Fischer wrote:
>>>>>
>>>>>>On September 03, 2003 at 10:48:31, Vincent Diepeveen wrote:
>>>>>>
>>>>>>>>I only see the need for communication when there is *somthing* to communicate.
>>>>>>>
>>>>>>>You answer your own question already. There continuesly is something to
>>>>>>>communicate.
>>>>>>
>>>>>>Such as?
>>>>>>
>>>>>>Whatever it is maybe it can be redesigned by using a smarter message system.
>>>>>>
>>>>>>The parent thread doesn't need to know *what* the child thread is doing, it only
>>>>>>needs to know what the child threads finds, if anything at all, right?
>>>>>>
>>>>>>-S.
>>>>>
>>>>>The only one who you are confusing is yourself.
>>>>>
>>>>>DIEP runs fine at any latency, but the speedup simply gets a lot less when the
>>>>>latency goes up.
>>>>>
>>>>>There are many practical problems.
>>>>>
>>>>>You speak about shipping messages.
>>>>>
>>>>>When are you going to receive them. Check each millisecond?
>>>>>
>>>>>Or let the OS decide?
>>>>>
>>>>>The OS fires at 100Hz, so things like processes that are sleeping because of the
>>>>>OS putting them to sleep (when locking and for 600 times they can't get the
>>>>>lock) then you have a latency of 10 ms before the process is awake.
>>>>>
>>>>>You are aware of such problems?
>>>>>
>>>>
>>>>No, because there is no such problem.  If you are running something else on
>>>>the same CPU, then you will see that 10ms latency.  If that CPU is idle, then
>>>>the instant the process is unblocked it will begin execution.
>>>
>>>Wrong, the 10ms latency is there to put something in the RUN queue of the
>>>kernel. Though there is no technical reason to remove that 10ms for the OS
>>>programmers, they are not allowed to do that, because that is violating
>>>agreements with important software manufacturers which have written software
>>>that assumes 10ms latency here and this crucial software will crash and cause
>>>severe problems if it is no longer there.
>>>
>>>The OS helpdesk.
>>
>>It absolutely does _not_ work like that.  What happens is this:
>>
>>Processes are blocked.  As an interrupt comes in, a process gets moved from
>>blocked to ready.  The temptation is to move that process from ready to run
>>if it is higher in priority than the process already in run.  But that causes
>>excessive context switching.  So, the process gets moved to ready and there
>>it sits until the next 10ms timer interrupt fires, and _then_ the scheduler
>>is called to move the currently running process back to ready, and the newly
>>ready process (of a higher priority) into running.
>>
>>That is _all_ there is to it.
>>
>>If the CPU is idle, and the interrupt comes in, the process is scheduled
>>_right now_, it goes from blocked to ready to run _instantly_.  No 10ms
>>delay.
>>
>>There is no doubt about how that works.  And your explanation is simply
>>garbage.  Ask some of the linux kernel guys.  Ingo Molnar is a good one to
>>ask although Alan Cox will also answer.
>
>The double origin3800 has 1024 cpu's. One partition (P7) is sized 512 processors
>from which 500 can get used simultaneously to run a single program cc-NUMA.

now one more thing to mention here. Please don't start saying how good linux is
compared to other OSes.

For cc-NUMA it's a joke simply. It's performing for latency and scheduling very
very poor when compared to IRIX.

From all OSes at any parallel machine i would blindfolded pick IRIX for
performance.

Linux has a long way to go and will never reach that. GCC never did either with
no hope of reaching it.

I'm not part of the linux kernel but i have been following the different
movements in the linux world for quite some years now. Both linux the OS and the
compiler have 1 big advantage and that's portability because it is not hardware
dependant.

That is directly its big disadvantage too. At cc-NUMA machines taking advantage
of the hardware architecture is very important and with gcc they never did much
effort to achieve that and the cc-NUMA scheduler of linux will be having that
destiny too.

So if someone says that SGI supercomputers with 64 processors and a linux kernel
perform very well and very stable, then that's not because of the linux kernel
in itself, but the modifications performed privately by SGI, officially called
'extension modifications' for trivial legal reasons, though i might not agree
legally with that.

Right now everything that's in the pipeline for kernel 2.6 is simply not enough
to work well cc-NUMA.

So don't praise the OS without knowing how poor it performs at cc-NUMA
platforms. I've been working a month now at a 64 processor linux kernel
2.4.20-220sgiR3 and it sucks ass.

to mention 1 concrete thing. When just 12 processors are in use eating 24GB ram
(machine has 2GB a processor) and another additional job is scheduled, then the
random latency increases by 40% when compared to a complete idle machine.

Under IRIX that isn't the case for 64 processor partitions.

>So sorry to inform you that Linux doesn't run on any partition cc-NUMA > 64
>cpu's.
>
>IRIX does though.
>
>With 10 braincells one could have known though that the SGI-Origin3800 is
>running IRIX. I did at least, that's why i asked SGI instead and got back this
>answer.
>
>In fact way more was answerred than just this.
>
>Best regards,
>Vincent
>
>>
>>>
>>>Thank you,
>>>Vincent
>>>
>>>>I have explained this before.  You can read about it in any good O/S book
>>>>that discusses this approach.  Your "conception" of the algorithm is flawed.
>>>>
>>>>>DIEP only can run well parallel at a shitload of cpu's thanks to statistical
>>>>>chances that a scenario X doesn't happen much.
>>>>>
>>>>>That took 1.2 years fulltime work. Still tuning some details.
>>>>>
>>>>>There is a lot of communication. It is very easy to test this yourself.
>>>>>
>>>>>Just get a cheap network card. Say 100mbit and connect 2 pc's. Now let them do a
>>>>>parallel search.
>>>>>
>>>>>Please report back to me when you have a speedup > 1.0, because initially you'll
>>>>>be slower than 1.0 i bet.
>>>>>
>>>>>Best regards,
>>>>>Vincent



This page took 0 seconds to execute

Last modified: Thu, 15 Apr 21 08:11:13 -0700

Current Computer Chess Club Forums at Talkchess. This site by Sean Mintz.