3 ms·

When you say: the real crime here is how awful Linux is at HPC without a lot of tuning and careful tweaking To what are you making the comparison? Are *BSD

When you say: the real crime here is how awful Linux is at HPC without a lot of tuning and careful tweaking To what are you making the comparison? Are *BSDs better? HP-Unix? AIX? Windows? What's the competition in this space that makes Linux look bad in HPC? Edit: Added "in HPC" at the end for clarification.

jsjohnst 4y ago
I’m curious if GP has any positive examples too, but did want to mention that being “the best of the awful” doesn’t make one not “awful”.
- pixl97 4y ago
  Typically being 'best of awful' means you have both a difficult problem and a very small dataset to work with where the solutions are not easily generalizable. You have to look at each and every HPC workload as a custom application where very small changes in your input and environment could have massive behavior changes in the application performance.
  - jsjohnst 4y ago
    I don’t disagree with you, but that doesn’t negate GP’s or mine’s point.
trelane 4y ago
I don't think there has to be a better, existing alternative to say something is terrible. AFAICT, this is just generally a hard space to get right, especially by default, on a system that will power both a data center - size supercomputer and system that is barely me then a microcontroller.
pjmlp 4y ago
The competition is naturally UNIX based OSes, given the historical background of such systems. Even the workloads that use GNU/Linux, most likely are heavily customized versions provided by IBM, HP and co. https://www.ibm.com/high-performance-computing https://www.ibm.com/high-performance-computing https://www.hpe.com/us/en/compute/hpc.html https://www.hpe.com/us/en/compute/hpc.html I also remember that for a while IBM's xl compilers were used quite often, not sure if that is still the case. See https://www.top500.org/ https://www.top500.org/
- gnufx 4y ago
  The competition probably shouldn't be full Unix, and in Blue Gene, for instance, it wasn't on the compute nodes. I don't know exactly what the current Cray environment is like -- it used to be rather odd -- but most HPC systems run normal EL-ish distributions. Summit is RHEL8, and mostly uses GCC, not XL according to https://gcc.gnu.org/wiki/cauldron2022talks?action=AttachFile&do=view&target=OpenMP-OpenACC-Offload-Cauldron2022-1.pdf https://gcc.gnu.org/wiki/cauldron2022talks?action=AttachFile...
  - pjmlp 4y ago
    Sure, however they are mostly UNIX inspired if you will, as anything else.
    - gnufx 4y ago
      Single-user, single-program doesn't seem very Unix-y to me: https://en.wikipedia.org/wiki/CNK_operating_system https://en.wikipedia.org/wiki/CNK_operating_system There was also Plan 9, but I don't know if it was ever actually used: https://doc.cat-v.org/plan_9/blue_gene/ https://doc.cat-v.org/plan_9/blue_gene/
convolvatron 4y ago
There has been an ongoing thread that claims that hpc workloads need no kernel or very little and that sharing resources doesn’t make sense at that grain size. This makes sense to me

jsjohnst 4y ago

I’m curious if GP has any positive examples too, but did want to mention that being “the best of the awful” doesn’t make one not “awful”.

pixl97 4y ago
Typically being 'best of awful' means you have both a difficult problem and a very small dataset to work with where the solutions are not easily generalizable. You have to look at each and every HPC workload as a custom application where very small changes in your input and environment could have massive behavior changes in the application performance.
- jsjohnst 4y ago
  I don’t disagree with you, but that doesn’t negate GP’s or mine’s point.

trelane 4y ago

I don't think there has to be a better, existing alternative to say something is terrible. AFAICT, this is just generally a hard space to get right, especially by default, on a system that will power both a data center - size supercomputer and system that is barely me then a microcontroller.

pjmlp 4y ago

The competition is naturally UNIX based OSes, given the historical background of such systems. Even the workloads that use GNU/Linux, most likely are heavily customized versions provided by IBM, HP and co. https://www.ibm.com/high-performance-computing https://www.ibm.com/high-performance-computing https://www.hpe.com/us/en/compute/hpc.html https://www.hpe.com/us/en/compute/hpc.html I also remember that for a while IBM's xl compilers were used quite often, not sure if that is still the case. See https://www.top500.org/ https://www.top500.org/

gnufx 4y ago
The competition probably shouldn't be full Unix, and in Blue Gene, for instance, it wasn't on the compute nodes. I don't know exactly what the current Cray environment is like -- it used to be rather odd -- but most HPC systems run normal EL-ish distributions. Summit is RHEL8, and mostly uses GCC, not XL according to https://gcc.gnu.org/wiki/cauldron2022talks?action=AttachFile&do=view&target=OpenMP-OpenACC-Offload-Cauldron2022-1.pdf https://gcc.gnu.org/wiki/cauldron2022talks?action=AttachFile...
- pjmlp 4y ago
  Sure, however they are mostly UNIX inspired if you will, as anything else.
  - gnufx 4y ago
    Single-user, single-program doesn't seem very Unix-y to me: https://en.wikipedia.org/wiki/CNK_operating_system https://en.wikipedia.org/wiki/CNK_operating_system There was also Plan 9, but I don't know if it was ever actually used: https://doc.cat-v.org/plan_9/blue_gene/ https://doc.cat-v.org/plan_9/blue_gene/

convolvatron 4y ago

There has been an ongoing thread that claims that hpc workloads need no kernel or very little and that sharing resources doesn’t make sense at that grain size. This makes sense to me