3 ms·

Not really, no. You've still got the overhead of copying, which can be significant. There's no way you could implement something like, for example, Gaussian eli

Not really, no. You've still got the overhead of copying, which can be significant. There's no way you could implement something like, for example, Gaussian elimination with message passing in a way that's just as fast as using locks and barriers.

greglindahl 9y ago
I totally agree that zzzcpan was over-generalizing, but so are you: is it really the case that the Linpack benchmark (which uses arrays as large as memory) on the latest SGI shared-memory thing uses locks and barriers?
- zzzcpan 9y ago
  I think the cost of locks is so high, that even some primitive runtime zero-copying mechanism won't add significant overhead. After all, if you have to do something on another core, you have to copy it there, it's not really a physical shared memory.
- openasocket 9y ago
  Maybe not locks and barriers specifically, but I'd be shocked if it's using message passing, particularly because of the overhead of copying. If anyone has evidence to the contrary I'd be happy to hear it.
  - greglindahl 9y ago
    Wow. I guess you aren't familiar with the history of large SGI Origin and Altix machines being mostly used as pure MPI machines because that's almost always fastest? Also, Linpack doesn't involve much data motion. Also, moving data over a shared memory interconnect is as much of a copy as moving data over a message passing interconnect. I was going to look up the current details for you, but I don't see an SGI Ultraviolet system in the current Top500. How large of a machine do you work with?
    - openasocket 9y ago
      I was thinking about machines with a shared memory bus (I'm 90% sure that's the right term) so yeah not things in the Top500. I'm thinking about your standard 64-core servers. I know that in the large MPI is best because getting that many cores to use the same memory bus just isn't feasible. Sorry for the confusion, I didn't read your comment carefully.

greglindahl 9y ago

I totally agree that zzzcpan was over-generalizing, but so are you: is it really the case that the Linpack benchmark (which uses arrays as large as memory) on the latest SGI shared-memory thing uses locks and barriers?

zzzcpan 9y ago
I think the cost of locks is so high, that even some primitive runtime zero-copying mechanism won't add significant overhead. After all, if you have to do something on another core, you have to copy it there, it's not really a physical shared memory.
openasocket 9y ago
Maybe not locks and barriers specifically, but I'd be shocked if it's using message passing, particularly because of the overhead of copying. If anyone has evidence to the contrary I'd be happy to hear it.
- greglindahl 9y ago
  Wow. I guess you aren't familiar with the history of large SGI Origin and Altix machines being mostly used as pure MPI machines because that's almost always fastest? Also, Linpack doesn't involve much data motion. Also, moving data over a shared memory interconnect is as much of a copy as moving data over a message passing interconnect. I was going to look up the current details for you, but I don't see an SGI Ultraviolet system in the current Top500. How large of a machine do you work with?
  - openasocket 9y ago
    I was thinking about machines with a shared memory bus (I'm 90% sure that's the right term) so yeah not things in the Top500. I'm thinking about your standard 64-core servers. I know that in the large MPI is best because getting that many cores to use the same memory bus just isn't feasible. Sorry for the confusion, I didn't read your comment carefully.