30 ms·

It's far too expensive with dubious impacts on forecast quality. Adaptive mesh approaches are far more suitable for high res global weather modeling... Why simu

It's far too expensive with dubious impacts on forecast quality. Adaptive mesh approaches are far more suitable for high res global weather modeling... Why simulate the area under boring, dynamically unimportant areas?

SergeAx 4y ago
At least three industries will be grateful for high-res wind predictions: aerospace, maritime and wind-generation.
Maursault 4y ago
> Why simulate the area under boring, dynamically unimportant areas? Butterflies... terrifying large, kilometer-scale butterflies.
- counters 4y ago
  Which is *exactly* why ultra-high resolution global weather simulation has dubious prospects for improving forecasts. When you're at spatial scales where you need to parameterize convection, there's an inherent "smoothness" to model solutions that suppresses noisy errors. If you go to cloud-resolving scales - which is needed for simulations like the ones here - you don't get the benefit of that smoothness anymore, because you need to actually resolve scales of motion that are incredibly fine. It's a losing proposition; you'll never get it "perfect", so you're much more likely to spin up an error cascade with significant impacts on forecast down the line, through things like the structure of organized convection. But dynamically uninteresting, quasi-balanced setups and modes? There's far less to worry about in terms of the butterfly effect, and any errors you might worry about will be dwarfed by the fact that we don't have good data to assimilate in places like the remote oceans anyways. It's also worth pointing out that the mathematics and understanding of error / perturbation growth in the atmosphere are well-understood. In fact, this fundamentally underpins how we've developed data assimilation approaches over the past two or three decades that allow us to effectively leverage new datasets such as satellite data to increase forecast quality and reliability at longer lead times. So it's somewhat trivial to actually directly quantify these "butterflies."
  - Maursault 4y ago
    If we're ever going to get to the femtometer resolution required for very precise 100 day weather forecasting, we have to start somewhere, so let them waste their time. It's not as though this is part of a growing trend to abandon conventional weather and climate modeling.
    - adgjlsfhk1 4y ago
      there's a very good physical argument that this is impossible. if you want to store 1 bit per femptometer simulated, at current computer sizes, we are taking about a computer billions the size of the earth. even if you use 1 atom per bit, your computer will be almost as big as the earth. such a computer will collapse under it's own gravity.
      - Maursault 4y ago
        > at current computer sizes This. No, not at all at current computer sizes, but at future computer sizes. This is the same mistake someone in the 1970's might make about billions having a smartphone today (supercomputer by their standards). Consider how everything at current computer sizes is effectively two dimensional, even stacked processors are still fundamentally 2D designs. There is still a lot of computing advancement ahead. 40 years from now they'll look back and think the same things we think when we look back 40 years, that the machines were so primitive, hardly anything could be done with them, and some will be nostalgic for them, talk about their strengths, while others will shake their heads and think even messing with the fastest workstation today is a waste of time. Just because we can't conceive of how, doesn't mean it's not possible, some day.
        adgjlsfhk1 4y ago
        do the math at 1 atom per bit.
    - counters 4y ago
      Why do you think that we need "femtometer resolution" for "very precise" 100 day weather forecasting? What even is "very precise" 100 day weather forecasting? I think it's very amusing to do the math on how much memory would be required to run a crude primitive equation dycore over even the tiniest of domains at femtometer resolution :) > It's not as though this is part of a growing trend to abandon conventional weather and climate modeling. The thing is, there *absolutely is* a trend towards private investment in weather modeling going towards faux-moonshot ideas like cubesat constellations without demonstrated ROI and that would require evolutionary leaps forward in data assimilation, or for deep learning to replace weather models. A miniature version of this already played out with precipitation nowcasting - probably the easiest weather forecasting problem that you could approach with an AI system, yet the approaches that have been developed so far barely improve over optical flow or other simple approaches, let alone advance our capability to forecast, say, convective initiation. The future of weather forecasting is larger ensembles (O(100-500) ensemble members, across 2-5 different models) of near-convective-resolving global models at meso-gamma (2-10 km resolution) fed into slightly more sophisticated statistical post-processing systems - almost certainly trained using simple AI/ML techniques on large-scale reforecasts of these parent model systems, or brute-forcing purely Bayesian statistical approaches.
- youainti 4y ago
  And the butterflies are full of hate. https://www.schlockmercenary.com/2017-07-13 https://www.schlockmercenary.com/2017-07-13
  - Maursault 4y ago
    As I have reluctantly learned today.

SergeAx 4y ago

At least three industries will be grateful for high-res wind predictions: aerospace, maritime and wind-generation.

Maursault 4y ago

> Why simulate the area under boring, dynamically unimportant areas? Butterflies... terrifying large, kilometer-scale butterflies.

counters 4y ago
Which is *exactly* why ultra-high resolution global weather simulation has dubious prospects for improving forecasts. When you're at spatial scales where you need to parameterize convection, there's an inherent "smoothness" to model solutions that suppresses noisy errors. If you go to cloud-resolving scales - which is needed for simulations like the ones here - you don't get the benefit of that smoothness anymore, because you need to actually resolve scales of motion that are incredibly fine. It's a losing proposition; you'll never get it "perfect", so you're much more likely to spin up an error cascade with significant impacts on forecast down the line, through things like the structure of organized convection. But dynamically uninteresting, quasi-balanced setups and modes? There's far less to worry about in terms of the butterfly effect, and any errors you might worry about will be dwarfed by the fact that we don't have good data to assimilate in places like the remote oceans anyways. It's also worth pointing out that the mathematics and understanding of error / perturbation growth in the atmosphere are well-understood. In fact, this fundamentally underpins how we've developed data assimilation approaches over the past two or three decades that allow us to effectively leverage new datasets such as satellite data to increase forecast quality and reliability at longer lead times. So it's somewhat trivial to actually directly quantify these "butterflies."
- Maursault 4y ago
  If we're ever going to get to the femtometer resolution required for very precise 100 day weather forecasting, we have to start somewhere, so let them waste their time. It's not as though this is part of a growing trend to abandon conventional weather and climate modeling.
  - adgjlsfhk1 4y ago
    there's a very good physical argument that this is impossible. if you want to store 1 bit per femptometer simulated, at current computer sizes, we are taking about a computer billions the size of the earth. even if you use 1 atom per bit, your computer will be almost as big as the earth. such a computer will collapse under it's own gravity.
    - Maursault 4y ago
      > at current computer sizes This. No, not at all at current computer sizes, but at future computer sizes. This is the same mistake someone in the 1970's might make about billions having a smartphone today (supercomputer by their standards). Consider how everything at current computer sizes is effectively two dimensional, even stacked processors are still fundamentally 2D designs. There is still a lot of computing advancement ahead. 40 years from now they'll look back and think the same things we think when we look back 40 years, that the machines were so primitive, hardly anything could be done with them, and some will be nostalgic for them, talk about their strengths, while others will shake their heads and think even messing with the fastest workstation today is a waste of time. Just because we can't conceive of how, doesn't mean it's not possible, some day.
      - adgjlsfhk1 4y ago
        do the math at 1 atom per bit.
  - counters 4y ago
    Why do you think that we need "femtometer resolution" for "very precise" 100 day weather forecasting? What even is "very precise" 100 day weather forecasting? I think it's very amusing to do the math on how much memory would be required to run a crude primitive equation dycore over even the tiniest of domains at femtometer resolution :) > It's not as though this is part of a growing trend to abandon conventional weather and climate modeling. The thing is, there *absolutely is* a trend towards private investment in weather modeling going towards faux-moonshot ideas like cubesat constellations without demonstrated ROI and that would require evolutionary leaps forward in data assimilation, or for deep learning to replace weather models. A miniature version of this already played out with precipitation nowcasting - probably the easiest weather forecasting problem that you could approach with an AI system, yet the approaches that have been developed so far barely improve over optical flow or other simple approaches, let alone advance our capability to forecast, say, convective initiation. The future of weather forecasting is larger ensembles (O(100-500) ensemble members, across 2-5 different models) of near-convective-resolving global models at meso-gamma (2-10 km resolution) fed into slightly more sophisticated statistical post-processing systems - almost certainly trained using simple AI/ML techniques on large-scale reforecasts of these parent model systems, or brute-forcing purely Bayesian statistical approaches.
youainti 4y ago
And the butterflies are full of hate. https://www.schlockmercenary.com/2017-07-13 https://www.schlockmercenary.com/2017-07-13
- Maursault 4y ago
  As I have reluctantly learned today.