3 ms·

This sounds like a good idea until you realize that your computations will be dominated by synchronization and/or communication overhead.

by readams 10y ago

mpweiher 10y ago
The point of the article was that there is so much extra parallel horsepower available that you would come out ahead even if overhead is something staggeringly ridiculous like 100x. That's what makes the idea interesting (though obviously not necessarily correct, it's just a thought-experiment).
- njloof 10y ago
  Ther Connection Machine was able to handle arbitrary n-processors-to-n-processors "sends" in (log2 n) time, but only by interconnecting the processors in a (log2 n)-dimensional hypercube. That's a lot of interconnects...
  - dfox 10y ago
    Also, Connection Machine does not have independent processors that execute arbitrary code and do uncoordinated communications. In traditional CM, everything happens synchronously under control of one global microcode sequencer, so communication can be planned by some offline algorithm that finds optimal paths for any given N to M communication operation. Also CM "processors" are not CPU's in any meaningful sense. Each CM's processor is essentially multiplexer and few bits of registers. Instruction is presented to the data inputs of multiplexer and processor state goes to the selection inputs with mux output being the next CPU state. In essence it's FPGA turned inside-out.
elihu 10y ago
Setting aside for a moment the wisdom of using thousands of cores to enable object oriented programming (which I'm dubious of), it seems to me that the way modern processors handle communication and synchronization is mostly a historical accident, and we could probably do a lot better than just letting processes share memory willy-nilly and having the processor cache sort it out. Like maybe we could give each core a set of registers dedicated explicitly for communicating with its neighbors, bypassing the memory hierarchy entirely for small messages.
- gpderetta 10y ago
  That didn't work very well for Cell.

This sounds like a good idea until you realize that your computations will be dominated by synchronization and/or communication overhead.

by readams 10y ago

This sounds like a good idea until you realize that your computations will be dominated by synchronization and/or communication overhead.

mpweiher 10y ago

The point of the article was that there is so much extra parallel horsepower available that you would come out ahead even if overhead is something staggeringly ridiculous like 100x. That's what makes the idea interesting (though obviously not necessarily correct, it's just a thought-experiment).

njloof 10y ago
Ther Connection Machine was able to handle arbitrary n-processors-to-n-processors "sends" in (log2 n) time, but only by interconnecting the processors in a (log2 n)-dimensional hypercube. That's a lot of interconnects...
- dfox 10y ago
  Also, Connection Machine does not have independent processors that execute arbitrary code and do uncoordinated communications. In traditional CM, everything happens synchronously under control of one global microcode sequencer, so communication can be planned by some offline algorithm that finds optimal paths for any given N to M communication operation. Also CM "processors" are not CPU's in any meaningful sense. Each CM's processor is essentially multiplexer and few bits of registers. Instruction is presented to the data inputs of multiplexer and processor state goes to the selection inputs with mux output being the next CPU state. In essence it's FPGA turned inside-out.

elihu 10y ago

Setting aside for a moment the wisdom of using thousands of cores to enable object oriented programming (which I'm dubious of), it seems to me that the way modern processors handle communication and synchronization is mostly a historical accident, and we could probably do a lot better than just letting processes share memory willy-nilly and having the processor cache sort it out. Like maybe we could give each core a set of registers dedicated explicitly for communicating with its neighbors, bypassing the memory hierarchy entirely for small messages.

gpderetta 10y ago
That didn't work very well for Cell.