5 ms·

Perlmutter is a new kind of supercomputer - It’s mini-cloud disguised as a supercomputer smaller DOE projects to funnel their compute to. Cori was kind of that

by prpl 5y ago

Perlmutter is a new kind of supercomputer - It’s mini-cloud disguised as a supercomputer smaller DOE projects to funnel their compute to. Cori was kind of that too, but IIRC kind of on accident because the Xeon Phi wasn’t available during delivery time, so the first “phase” was Haswell nodes. I’m not sure what this means for whole-machine reservation but hopefully that’s a thing of the past in any case. Ultimately this is a machine to solve everybody’s needs, not just ML needs (although it must solve those needs very well in any case) I’m not sure Cori will be remembered particularly fondly, but Perlmutter will probably be because it seems like it will be versatile enough to meet everyone’s needs. For those wondering why not cloud - most simulations generally make sense in cloud because there’s nowhere near the data movement/storage involved and anything in particle physics is always embarrassingly parallel. The data isn’t moving there though - it is very cheap to keep the data at a place like NERSC especially with tape in the mix.

dekhn 5y ago
One of the reasons I left DOE for industry many years ago was that so much of the funding went to large supercomputers with about 8 users (the folks who had made their code scale to the largest size) while vast majority of people with compute needs went unsupported. Tape is worthless. Tape is where you put data you never want to retrieve again.
- prpl 5y ago
  I understand most of what you are saying, and it isn’t pleasant in many cases but I think the situation is improving. Yes, all that happens, and most project computing (Office of Science) is usually pushed to NERSC, but it really depends on many factors. NERSC has a lot of users (much more than 8), probably the most out of the DOE complex, since it’s a user facility. Big projects (>$500M) don’t often have this problem and most of them will push for clusters they own at their home lab. Small projects are most likely to have trouble either in procuring resources or effectively using super computing resources. A dirty secret is many users just use MPI for a job queue/coordination so their jobs “scale” when large reservations are required, but IB is not really exercised in those situations. When you realize that, it’s easy to realize most workflows scale (as long as disk is considered), but very annoying from a development perspective. Tape isn’t worthless because you have to have a disaster plan or long term data preservation plan and it’s usually just a part of that, but some labs/projects have historically pushed as part of the integral data management plan for active/warm data and that is indeed worthless.
  - dekhn 5y ago
    I really doubt people are running embarassingly parallel codes on NERSC supercomputers. I was a PI at NERSC and I was told explicitly, repeatedly, I couldn't do that. That's why I moved to Open Science Grid, and eventually Google's compute platform. Nope, tape is just useless. Live but cold hard drives.

by prpl 5y ago

dekhn 5y ago

One of the reasons I left DOE for industry many years ago was that so much of the funding went to large supercomputers with about 8 users (the folks who had made their code scale to the largest size) while vast majority of people with compute needs went unsupported. Tape is worthless. Tape is where you put data you never want to retrieve again.

prpl 5y ago
I understand most of what you are saying, and it isn’t pleasant in many cases but I think the situation is improving. Yes, all that happens, and most project computing (Office of Science) is usually pushed to NERSC, but it really depends on many factors. NERSC has a lot of users (much more than 8), probably the most out of the DOE complex, since it’s a user facility. Big projects (>$500M) don’t often have this problem and most of them will push for clusters they own at their home lab. Small projects are most likely to have trouble either in procuring resources or effectively using super computing resources. A dirty secret is many users just use MPI for a job queue/coordination so their jobs “scale” when large reservations are required, but IB is not really exercised in those situations. When you realize that, it’s easy to realize most workflows scale (as long as disk is considered), but very annoying from a development perspective. Tape isn’t worthless because you have to have a disaster plan or long term data preservation plan and it’s usually just a part of that, but some labs/projects have historically pushed as part of the integral data management plan for active/warm data and that is indeed worthless.
- dekhn 5y ago
  I really doubt people are running embarassingly parallel codes on NERSC supercomputers. I was a PI at NERSC and I was told explicitly, repeatedly, I couldn't do that. That's why I moved to Open Science Grid, and eventually Google's compute platform. Nope, tape is just useless. Live but cold hard drives.