4 ms·

I'm excited on behalf of the scientists I know for cloud HPC. I see a lot of "develop program locally on tiny subset of data" followed by "how the hell am I ev

by ip26 5y ago

I'm excited on behalf of the scientists I know for cloud HPC. I see a lot of "develop program locally on tiny subset of data" followed by "how the hell am I ever going to run the full model for the final run on all the data!?" A couple hours machine time leased on a behemoth is really the ticket, and not really that expensive.

zozbot234 5y ago
The cost of data egress from the cloud really limits its potential for most HPC applications, though. The "free" or reasonable-cost tiers are okay for small-scale toy use, but not much more than that.
- AitchEmArsey 5y ago
  For academia, all 3 major cloud providers have a data egress waiver - meaning that as long as data egress is less than 15% of the total bill then it is free. Combine this with the fact that S3/Glacier make much more sense than local HDDs for long-term archiving and egress turns out not to be a significant factor in my experience.