8 ms·

Met Office to build £97m supercomputer

49 points by hereprompt 12y ago

QuadDamaged 12y ago
I misread and was really wondering what the Met Police would do with a GBP 97M supercomputer.
- hkmurakami 12y ago
  I totally thought this was Met Life Insurance, which would have been mildly plausible. ;)
justincormack 12y ago
Even the Met Office's press release just says it weights as much as 11 double decker buses rather than any actual technical details. I presume they didnt buy it by weight, why dumb everything down.
- justincormack 12y ago
  Ah Cray has some actual details[1]. [1] http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1982080 http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsA...
- quarterto 12y ago
  100 tons, according to the radio this morning.
- Graham24 12y ago
  The double decker bus is a standard unit of measurement for height and weight, for large areas it's Wales.
  - petepete 12y ago
    Medium areas are well-covered by football pitches.
    - scottmcf 12y ago
      Olympic sized swimming pools give a little more granularity.
- gjm11 12y ago
  To be fair, that press release does give - quantitative information about the new system's power relative to its predecessor ("13 times more powerful"; I assume they mean something like "13x faster on typical workloads") - quantitative information about its speed in absolute terms ("more than 16,000 trillion calculations per second") - quantitative information about what it will let them do that they couldn't before ("forecast updates every hour", "very high resolution (300m) models", and few particular projects they intend to use it for) - a little more tech-spec information, albeit stupidly presented ("120,000 times more memory than a top-end smartphone", which I presume means either 120TB or 240TB) - how much they're spending - how long they expect it to take to get it set up and working and they do give the weight in tonnes as well as in double-decker buses. I don't really see that there's much of a problem here.
  - deleted 12y ago
    [deleted]
  - tobylane 12y ago
    To compare to the top supercomputers, Cray Titan cost around $160m at the end of 2013 and has 17 petaflops. Met Office also use Cray, so it might be that they ordered Titan with a little less flops, a lot less memory.
qewrffewqwfqew 12y ago
> The facility will work 13 times faster than the current system, enabling detailed, UK-wide forecast models with a resolution of 1.5km to be run every single hour, rather than every three. Let's say an order of magnitude speed improvement. :wow: I wonder if it will be able to model the local microclimate effect of powering and cooling 140 tonnes of compute?
jmngomes 12y ago
Glad they now have a faster way to make inaccurate predictions :)
- druml 12y ago
  Computing power actually matters for the accuracy of weather prediction. Having more computing power would allow them to refine the spatial and temporal resolution of their weather forecast model. http://en.wikipedia.org/wiki/Numerical_weather_prediction#Computation http://en.wikipedia.org/wiki/Numerical_weather_prediction#Co...
lordnacho 12y ago
What's a good source for learning how weather models work? I'm interested in understanding how the accuracy changes with the number of calculations available.
nextw33k 12y ago
I would love to see the comparison between running this and just buying cloud compute time. Given its a constant operation which probably requires high speed interconnects. However it would be interesting to see how close in cost cloud is to a bare metal build.
- philjohn 12y ago
  here's a hint - it's nowhere near close in cost to a bare metal build, at least not in my experience. We run a small clustered database across 8 nodes. Total cost for the hardware was ~ $3000 per node. Cost for a comparable node in EC2 per month? $1500. Even with co-lo costs for power and bandwidth, and buying 10gige switches it's far cheaper for us to host it in house than it is to use EC2.
  - JoeAltmaier 12y ago
    How about the cost of bricks and mortar? The cost of an administrator? The cost of spares and disaster recovery? Sure the purchase cost looks good, but what about the total cost of ownership?
    - lucian1900 12y ago
      It's still tiny compared to AWS, which is really expensive. That doesn't mean it's not useful, but it's still tens of times more expensive than using your own hardware.
      - JoeAltmaier 12y ago
        Often total cost of ownership is about the same as the equipment cost. So they start to look comparable actually.
        lucian1900 12y ago
        That's not what I've found. Even just renting dedicated physical machines has been significantly cheaper overall.
Igglyboo 12y ago
>The facility will work 13 times faster than the current system, enabling detailed, UK-wide forecast models with a resolution of 1.5km to be run every single hour, rather than every three. Maybe I'm missing something but should a 13x speedup allow it to run in 1/13th the time? Or maybe the "13x" is only for part of the system and they have a bottleneck somewhere else.
- jgrahamc 12y ago
  See: http://en.wikipedia.org/wiki/Amdahl's_law http://en.wikipedia.org/wiki/Amdahl's_law
  - Igglyboo 12y ago
    Yes but they said 13x speedup not 13x more processors/cores. Probably just a non-technical writeup tbh.
- sp332 12y ago
  Maybe there's not much point in running it more than once an hour. For example, maybe the forecast is only reported once an hour. Edit: A quote later in the article says "It will allow us to add more precision, more detail, more accuracy". So it's probably increasing the resolution even though the part you quoted sounds like it's not.
jimmytidey 12y ago
Literal cloud computing...
bhouston 12y ago
Is it worth it to buy supercomputers these days or just rent cloud time? I guess it you need it all the time, this may be the way to go, but couldn't you get a bulk rate from Heitzner or OVH?
- micro_cam 12y ago
  Last time I did the math owning your own hardware was still significantly cheaper especially if you're an institution that already has some of the infrastructure in place (and covered by overhead costs on grants). And this is weather simulation so stuff like network topology matters and they will be running their models as often as they can. The cloud wins for more elastic work loads and people without IT departments and server rooms.
- saosebastiao 12y ago
  If cloud operators provided Fiber Channel networking, they would probably consume most of the flex capacity of those that need supercomputers. One example I know of (for weather simulations no less) has >100 tflops capacity, but their utilization is about 30% because they have little ability to level load for their customers.

49 points by hereprompt 12y ago

QuadDamaged 12y ago

I misread and was really wondering what the Met Police would do with a GBP 97M supercomputer.

hkmurakami 12y ago
I totally thought this was Met Life Insurance, which would have been mildly plausible. ;)

justincormack 12y ago

Even the Met Office's press release just says it weights as much as 11 double decker buses rather than any actual technical details. I presume they didnt buy it by weight, why dumb everything down.

justincormack 12y ago
Ah Cray has some actual details[1]. [1] http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsArticle&ID=1982080 http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsA...
quarterto 12y ago
100 tons, according to the radio this morning.
Graham24 12y ago
The double decker bus is a standard unit of measurement for height and weight, for large areas it's Wales.
- petepete 12y ago
  Medium areas are well-covered by football pitches.
  - scottmcf 12y ago
    Olympic sized swimming pools give a little more granularity.
gjm11 12y ago
To be fair, that press release does give - quantitative information about the new system's power relative to its predecessor ("13 times more powerful"; I assume they mean something like "13x faster on typical workloads") - quantitative information about its speed in absolute terms ("more than 16,000 trillion calculations per second") - quantitative information about what it will let them do that they couldn't before ("forecast updates every hour", "very high resolution (300m) models", and few particular projects they intend to use it for) - a little more tech-spec information, albeit stupidly presented ("120,000 times more memory than a top-end smartphone", which I presume means either 120TB or 240TB) - how much they're spending - how long they expect it to take to get it set up and working and they do give the weight in tonnes as well as in double-decker buses. I don't really see that there's much of a problem here.
- deleted 12y ago
  [deleted]
- tobylane 12y ago
  To compare to the top supercomputers, Cray Titan cost around $160m at the end of 2013 and has 17 petaflops. Met Office also use Cray, so it might be that they ordered Titan with a little less flops, a lot less memory.

qewrffewqwfqew 12y ago

> The facility will work 13 times faster than the current system, enabling detailed, UK-wide forecast models with a resolution of 1.5km to be run every single hour, rather than every three. Let's say an order of magnitude speed improvement. :wow: I wonder if it will be able to model the local microclimate effect of powering and cooling 140 tonnes of compute?

jmngomes 12y ago

Glad they now have a faster way to make inaccurate predictions :)

druml 12y ago
Computing power actually matters for the accuracy of weather prediction. Having more computing power would allow them to refine the spatial and temporal resolution of their weather forecast model. http://en.wikipedia.org/wiki/Numerical_weather_prediction#Computation http://en.wikipedia.org/wiki/Numerical_weather_prediction#Co...

lordnacho 12y ago

What's a good source for learning how weather models work? I'm interested in understanding how the accuracy changes with the number of calculations available.

nextw33k 12y ago

I would love to see the comparison between running this and just buying cloud compute time. Given its a constant operation which probably requires high speed interconnects. However it would be interesting to see how close in cost cloud is to a bare metal build.

philjohn 12y ago
here's a hint - it's nowhere near close in cost to a bare metal build, at least not in my experience. We run a small clustered database across 8 nodes. Total cost for the hardware was ~ $3000 per node. Cost for a comparable node in EC2 per month? $1500. Even with co-lo costs for power and bandwidth, and buying 10gige switches it's far cheaper for us to host it in house than it is to use EC2.
- JoeAltmaier 12y ago
  How about the cost of bricks and mortar? The cost of an administrator? The cost of spares and disaster recovery? Sure the purchase cost looks good, but what about the total cost of ownership?
  - lucian1900 12y ago
    It's still tiny compared to AWS, which is really expensive. That doesn't mean it's not useful, but it's still tens of times more expensive than using your own hardware.
    - JoeAltmaier 12y ago
      Often total cost of ownership is about the same as the equipment cost. So they start to look comparable actually.
      - lucian1900 12y ago
        That's not what I've found. Even just renting dedicated physical machines has been significantly cheaper overall.

Igglyboo 12y ago

>The facility will work 13 times faster than the current system, enabling detailed, UK-wide forecast models with a resolution of 1.5km to be run every single hour, rather than every three. Maybe I'm missing something but should a 13x speedup allow it to run in 1/13th the time? Or maybe the "13x" is only for part of the system and they have a bottleneck somewhere else.

jgrahamc 12y ago
See: http://en.wikipedia.org/wiki/Amdahl's_law http://en.wikipedia.org/wiki/Amdahl's_law
- Igglyboo 12y ago
  Yes but they said 13x speedup not 13x more processors/cores. Probably just a non-technical writeup tbh.
sp332 12y ago
Maybe there's not much point in running it more than once an hour. For example, maybe the forecast is only reported once an hour. Edit: A quote later in the article says "It will allow us to add more precision, more detail, more accuracy". So it's probably increasing the resolution even though the part you quoted sounds like it's not.

jimmytidey 12y ago

Literal cloud computing...

bhouston 12y ago

Is it worth it to buy supercomputers these days or just rent cloud time? I guess it you need it all the time, this may be the way to go, but couldn't you get a bulk rate from Heitzner or OVH?

micro_cam 12y ago
Last time I did the math owning your own hardware was still significantly cheaper especially if you're an institution that already has some of the infrastructure in place (and covered by overhead costs on grants). And this is weather simulation so stuff like network topology matters and they will be running their models as often as they can. The cloud wins for more elastic work loads and people without IT departments and server rooms.
saosebastiao 12y ago
If cloud operators provided Fiber Channel networking, they would probably consume most of the flex capacity of those that need supercomputers. One example I know of (for weather simulations no less) has >100 tflops capacity, but their utilization is about 30% because they have little ability to level load for their customers.