7 ms·

18 hours, $33K, and 156,314 cores: Amazon cloud HPC hits a “petaflop”

65 points by tiernano 13y ago

ChuckMcM 13y ago
"There are whole categories of problems that are pleasantly parallel, and in those cases the [Linpack maximum] number is really not as important because we did intentionally make use of the entire 1.2 petaflops because they were all concurrently executing the workloads," Stowe told Ars. "Maybe there does need to be a different metric for analytics and big data and genomics—and all these pleasantly parallel workloads that are becoming more pervasive." I wonder if people miss this part. For easily parallel projects, Amazon is great since any core can make progress regardless of the other cores state or reachability. But if there is any inter-dependence between the cores (I use the term engtangling but others have called it co-dependence) the number becomes limited by the bisection bandwidth between cores. Poorly scheduled (like randomly scheduling on a core) is the worst case, if you can keep the bandwidth between entangled cores high, even if the system bandwidth is low, you can improve things (dramatically some times). But at the end of the day its the level of sharing between cores that will tell you if you can do your problem in the 'cloud' or if you are going to need a local data center.
- deleted 13y ago
  [deleted]
codex 13y ago
See also: https://news.ycombinator.com/item?id=6720969 https://news.ycombinator.com/item?id=6720969 for the Google cloud version.
VladRussian2 13y ago
> including Amazon's cc2.8xlarge instance with 32 cores they count AWS "vCores" which is 1 HT ( half-core).
cj 13y ago
What's the advantage of running 150k cores for 18 hours versus far fewer cores over a period of a few days or weeks? Is efficiency gained with increased concurrency while processing data in a certain way?
- res0nat0r 13y ago
  Since it is massively parallel you are saving time.
- ceejayoz 13y ago
  If you're going to spend the same amount either way, you might as well get the results back in a day.
- njharman 13y ago
  Um, done in 18 hours rather than days or weeks is (or can be) a huge advantage in it's self.
- Fomite 13y ago
  You can get your answer back in 18 hours. For many research questions, being able to answer quickly has value - either for the use of the results, or if you're actually doing many runs doing parameter sweeps and the like.
wuschel 13y ago
Theory is one thing, hands-on experimental scientific evidence another, not speaking about a possible route of synthesis and purification of polyaromatic cyclic hydrocarbons. Does anyone know what problem/model was used for the calculations? I find it strange that there are many details about the hardware, but no information about the problem set in the field of organic solar cells.
- res0nat0r 13y ago
  Cycle is a HPC company that configures the software needed for these massively parallel computations, they don't actually do the protein modeling or any of the scientific research. That is all supplied from the customer. They basically setup the environment for the customer to massively "crunch the numbers."
arjie 13y ago
Interesting to see that 'flops' has been redefined to 'flop' so that 'flops' now refers to the plural. Also interesting to see that arstechnica hijacks my back button. Anyway, the real problem from what I've read in the literature is in problems that require network IO, but going back now, most of that seems to predate Amazon's HPC cloud service and even a year ago, some people have found pretty good scaling there: http://www.computer.org/portal/web/computingnow/content?g=53319&type=article&urlTitle=massively-parallel-fluid-simulations-on-amazon-s-hpc-cloud http://www.computer.org/portal/web/computingnow/content?g=53....
rlawton 13y ago
Very cool benchmark.

65 points by tiernano 13y ago

"There are whole categories of problems that are pleasantly parallel, and in those cases the [Linpack maximum] number is really not as important because we did intentionally make use of the entire 1.2 petaflops because they were all concurrently executing the workloads," Stowe told Ars. "Maybe there does need to be a different metric for analytics and big data and genomics—and all these pleasantly parallel workloads that are becoming more pervasive." I wonder if people miss this part. For easily parallel projects, Amazon is great since any core can make progress regardless of the other cores state or reachability. But if there is any inter-dependence between the cores (I use the term engtangling but others have called it co-dependence) the number becomes limited by the bisection bandwidth between cores. Poorly scheduled (like randomly scheduling on a core) is the worst case, if you can keep the bandwidth between entangled cores high, even if the system bandwidth is low, you can improve things (dramatically some times). But at the end of the day its the level of sharing between cores that will tell you if you can do your problem in the 'cloud' or if you are going to need a local data center.

deleted 13y ago
[deleted]

codex 13y ago

See also: https://news.ycombinator.com/item?id=6720969 https://news.ycombinator.com/item?id=6720969 for the Google cloud version.

VladRussian2 13y ago

> including Amazon's cc2.8xlarge instance with 32 cores they count AWS "vCores" which is 1 HT ( half-core).

cj 13y ago

What's the advantage of running 150k cores for 18 hours versus far fewer cores over a period of a few days or weeks? Is efficiency gained with increased concurrency while processing data in a certain way?

res0nat0r 13y ago
Since it is massively parallel you are saving time.
ceejayoz 13y ago
If you're going to spend the same amount either way, you might as well get the results back in a day.
njharman 13y ago
Um, done in 18 hours rather than days or weeks is (or can be) a huge advantage in it's self.
Fomite 13y ago
You can get your answer back in 18 hours. For many research questions, being able to answer quickly has value - either for the use of the results, or if you're actually doing many runs doing parameter sweeps and the like.

wuschel 13y ago

Theory is one thing, hands-on experimental scientific evidence another, not speaking about a possible route of synthesis and purification of polyaromatic cyclic hydrocarbons. Does anyone know what problem/model was used for the calculations? I find it strange that there are many details about the hardware, but no information about the problem set in the field of organic solar cells.

res0nat0r 13y ago
Cycle is a HPC company that configures the software needed for these massively parallel computations, they don't actually do the protein modeling or any of the scientific research. That is all supplied from the customer. They basically setup the environment for the customer to massively "crunch the numbers."

arjie 13y ago

Interesting to see that 'flops' has been redefined to 'flop' so that 'flops' now refers to the plural. Also interesting to see that arstechnica hijacks my back button. Anyway, the real problem from what I've read in the literature is in problems that require network IO, but going back now, most of that seems to predate Amazon's HPC cloud service and even a year ago, some people have found pretty good scaling there: http://www.computer.org/portal/web/computingnow/content?g=53319&type=article&urlTitle=massively-parallel-fluid-simulations-on-amazon-s-hpc-cloud http://www.computer.org/portal/web/computingnow/content?g=53....

rlawton 13y ago

Very cool benchmark.