4 ms·

Out of curiosity, I wonder how it compares with computer power used for ad placement prediction.

by tonfa 10y ago

semi-extrinsic 10y ago
Weather forecasting for Norway keeps about 500 dual-socket Xeon E5-2630 servers busy 24/7 all year. Norway has 0.3% of the world's land area, so a rough estimate, assuming Norway uses 5x more CPU-hours per sq. km. than the world average, would put global weather prediction use at about 30 000 nodes running 24/7. For comparison, a dedicated AWS m4.4xlarge instance, which is equivalent to one of these nodes without the expensive Infiniband, is about $500/month. For an actual supercomputer node you can figure about double that. So the total running cost (of just the hardware) for global weather prediction per month is probably around 30 million USD. Then you also have to pay people to manage the running of simulations, do the analysis, produce nice plots and animations for TV, etc.
- detaro 10y ago
  Can you scale the prediction effort like this? Don't they run a global model with just a focus on the "home area", or are the resolution differences so large that the "rest of the world" just isn't very expensive to calculate? Or are there shared global models that are used as input data?
  - semi-extrinsic 10y ago
    Those are very good questions. My post above attempts napkin math, so it could probably be off by a factor of 10, but probably not by a factor of 100. Also, I Am Not A Weather Forecast Scientist, just working in a related field (fluid dynamics). So I could be wrong about stuff here. As for shared models and data: I know Norway and the UK do a lot of this. I imagine so does the US and Canada/Mexico. Also, keep in mind that weather modelling relies heavily on running an ensemble with random variations in initial/boundary conditions. So a lot of the computational cost comes from running hundreds of these simulations. My guess is that you don't run the large area 100 times, just a few times and then perturb it for all your local area simulations.
vtuulos 10y ago
Fun question, even if it wasn't a serious one. Since I happen to work at a company whose business is related to ad placement prediction, I can try to answer. In our case, we have 1000-2000 instances (the number is elastic, because AWS) handling real-time ad auctions and related data processing. Since we are just one of the many players in the industry, and far smaller than the largest ones (GOOG, FB), I am pretty sure in aggregate more computing cycles are used to optimize ads than oil or weather. Crazy.

Out of curiosity, I wonder how it compares with computer power used for ad placement prediction.

by tonfa 10y ago

Out of curiosity, I wonder how it compares with computer power used for ad placement prediction.

Weather forecasting for Norway keeps about 500 dual-socket Xeon E5-2630 servers busy 24/7 all year. Norway has 0.3% of the world's land area, so a rough estimate, assuming Norway uses 5x more CPU-hours per sq. km. than the world average, would put global weather prediction use at about 30 000 nodes running 24/7. For comparison, a dedicated AWS m4.4xlarge instance, which is equivalent to one of these nodes without the expensive Infiniband, is about $500/month. For an actual supercomputer node you can figure about double that. So the total running cost (of just the hardware) for global weather prediction per month is probably around 30 million USD. Then you also have to pay people to manage the running of simulations, do the analysis, produce nice plots and animations for TV, etc.

detaro 10y ago
Can you scale the prediction effort like this? Don't they run a global model with just a focus on the "home area", or are the resolution differences so large that the "rest of the world" just isn't very expensive to calculate? Or are there shared global models that are used as input data?
- semi-extrinsic 10y ago
  Those are very good questions. My post above attempts napkin math, so it could probably be off by a factor of 10, but probably not by a factor of 100. Also, I Am Not A Weather Forecast Scientist, just working in a related field (fluid dynamics). So I could be wrong about stuff here. As for shared models and data: I know Norway and the UK do a lot of this. I imagine so does the US and Canada/Mexico. Also, keep in mind that weather modelling relies heavily on running an ensemble with random variations in initial/boundary conditions. So a lot of the computational cost comes from running hundreds of these simulations. My guess is that you don't run the large area 100 times, just a few times and then perturb it for all your local area simulations.

vtuulos 10y ago

Fun question, even if it wasn't a serious one. Since I happen to work at a company whose business is related to ad placement prediction, I can try to answer. In our case, we have 1000-2000 instances (the number is elastic, because AWS) handling real-time ad auctions and related data processing. Since we are just one of the many players in the industry, and far smaller than the largest ones (GOOG, FB), I am pretty sure in aggregate more computing cycles are used to optimize ads than oil or weather. Crazy.