3 ms·
Mimic sure, but it won't be quite the same. I won't deny you can't do a one off job possibly cheaper on aws. The thing is, these simulations run for weeks or mo
by mitchty 11y ago
Mimic sure, but it won't be quite the same. I won't deny you can't do a one off job possibly cheaper on aws. The thing is, these simulations run for weeks or months on end and these clusters are run and setup to run at 100% at all times.
They really are efficient power wise, which is the true test for things like this. I'm hitting towards nda things but needless to say keep an eye out for gpu technology in supercomputers. As well as what that top 500 chinese supercomputer has.
Note, its at the top mostly due to its accelerator cards. Most simulations don't need those.
I know 60 mill seems like a lot but when it runs for 5 years it really isn't that expensive. For a one off job sure go with aws, it'll take longer, but i'm not kidding when you get down to the detail of the cpu architecture and keep track of things like how cache misses affect a program.
It really is night and day difference in how things run. The avx comment is not a joke, reducing even cpu cost by 5% over months is a huge deal. These are just a few of the differences in how supercomputers operate. It is far less to do with the cpu and memory and more how they communicate amongst themselves. One node may compute a value, but another adjacent node then takes on that output to calculate more. This is the key differentiator, and if you aren't compiling fortran (not joking), its likely its not a very great simulation.
If you'd like to know more, i'd recommend getting a job at a supercomputer company. They've solved a lot of the overall cluster problems aws has many years ago. And it is really fun stuff where details matter. Something I can't say about most aws deployments. PS: I'm working on something that will make Titan look like a kid. :) Every 2-3 years this stuff gets pushed aside its fun.
- lmeyerov 11y agoWe handwrite PTX ;-) Cost-wise, public cloud destroys traditional supercomputers, and the gap will just keep increasing. However, the economics enabling it, e.g., sufficiently non-interfering multitenancy, are only now reaching the needs of HPC. Likewise, only recently did public clouds embrace non-wimpy hardware: SSDs, multiGPU, starting to have FPGAs, etc. Exposed interconnects are 10 gige, so not the best, but like you said, scale requires designing for minimizing communication to beginwith. RE:long jobs, we focus on interactive exploration (< 100ms). Spark etc. give up latency to ensure resiliency on long jobs.