6 ms·

Which is exactly why ultra-high resolution global weather simulation has dubious prospects for improving forecasts. When you're at spatial scales where you ne

Which is *exactly* why ultra-high resolution global weather simulation has dubious prospects for improving forecasts. When you're at spatial scales where you need to parameterize convection, there's an inherent "smoothness" to model solutions that suppresses noisy errors. If you go to cloud-resolving scales - which is needed for simulations like the ones here - you don't get the benefit of that smoothness anymore, because you need to actually resolve scales of motion that are incredibly fine. It's a losing proposition; you'll never get it "perfect", so you're much more likely to spin up an error cascade with significant impacts on forecast down the line, through things like the structure of organized convection. But dynamically uninteresting, quasi-balanced setups and modes? There's far less to worry about in terms of the butterfly effect, and any errors you might worry about will be dwarfed by the fact that we don't have good data to assimilate in places like the remote oceans anyways. It's also worth pointing out that the mathematics and understanding of error / perturbation growth in the atmosphere are well-understood. In fact, this fundamentally underpins how we've developed data assimilation approaches over the past two or three decades that allow us to effectively leverage new datasets such as satellite data to increase forecast quality and reliability at longer lead times. So it's somewhat trivial to actually directly quantify these "butterflies."

Maursault 4y ago
If we're ever going to get to the femtometer resolution required for very precise 100 day weather forecasting, we have to start somewhere, so let them waste their time. It's not as though this is part of a growing trend to abandon conventional weather and climate modeling.
- adgjlsfhk1 4y ago
  there's a very good physical argument that this is impossible. if you want to store 1 bit per femptometer simulated, at current computer sizes, we are taking about a computer billions the size of the earth. even if you use 1 atom per bit, your computer will be almost as big as the earth. such a computer will collapse under it's own gravity.
  - Maursault 4y ago
    > at current computer sizes This. No, not at all at current computer sizes, but at future computer sizes. This is the same mistake someone in the 1970's might make about billions having a smartphone today (supercomputer by their standards). Consider how everything at current computer sizes is effectively two dimensional, even stacked processors are still fundamentally 2D designs. There is still a lot of computing advancement ahead. 40 years from now they'll look back and think the same things we think when we look back 40 years, that the machines were so primitive, hardly anything could be done with them, and some will be nostalgic for them, talk about their strengths, while others will shake their heads and think even messing with the fastest workstation today is a waste of time. Just because we can't conceive of how, doesn't mean it's not possible, some day.
    - adgjlsfhk1 4y ago
      do the math at 1 atom per bit.
- counters 4y ago
  Why do you think that we need "femtometer resolution" for "very precise" 100 day weather forecasting? What even is "very precise" 100 day weather forecasting? I think it's very amusing to do the math on how much memory would be required to run a crude primitive equation dycore over even the tiniest of domains at femtometer resolution :) > It's not as though this is part of a growing trend to abandon conventional weather and climate modeling. The thing is, there *absolutely is* a trend towards private investment in weather modeling going towards faux-moonshot ideas like cubesat constellations without demonstrated ROI and that would require evolutionary leaps forward in data assimilation, or for deep learning to replace weather models. A miniature version of this already played out with precipitation nowcasting - probably the easiest weather forecasting problem that you could approach with an AI system, yet the approaches that have been developed so far barely improve over optical flow or other simple approaches, let alone advance our capability to forecast, say, convective initiation. The future of weather forecasting is larger ensembles (O(100-500) ensemble members, across 2-5 different models) of near-convective-resolving global models at meso-gamma (2-10 km resolution) fed into slightly more sophisticated statistical post-processing systems - almost certainly trained using simple AI/ML techniques on large-scale reforecasts of these parent model systems, or brute-forcing purely Bayesian statistical approaches.
  - Maursault 4y ago
    > Why do you think that we need "femtometer resolution" for "very precise" 100 day weather forecasting? Due to sensitive dependence on initial conditons. Even using measurements at meter resolution will cause the accuracy of a forecast to begin to break down after only a few days. > What even is "very precise" 100 day weather forecasting? Anywhere from accurate to exact. > I think it's very amusing to do the math on how much memory would be required to run a crude primitive equation dycore over even the tiniest of domains at femtometer resolution And Bill Gates thought 64K should be enough for anybody. Do you really think computers will only have a few GB of memory 50 years from now? > there absolutely is a trend towards private investment in weather modeling going towards faux-moonshot ideas like cubesat constellations without demonstrated ROI and that would require evolutionary leaps forward in data assimilation, or for deep learning to replace weather models This straw man does not exactly demonstrate that conventional weather and climate modeling is being abandoned anytime soon. If the unconventional private investments aren't profitable, the market will deal with them. > The future of weather forecasting is much like the local weather, impossible to predict with any accuracy years into the future, and yet the tools used to measure it are consistently getting more accurate, cheaper and smaller. Maybe like bottle-openers, weather sensors may superfluously start appear on everything. The more widespread the measurements, the more data descibing initial conditions, the better the forecast will be at any interval.
- contravariant 4y ago
  You want to know the precise shape of the Earth's surface in femtometer precision? There are some profound problems with that idea once you get below 10 meter or so, but I'll let you think that one through yourself.
  - Maursault 4y ago
    No, but I wouldn't mind weather measuments every cubic femtometer of the lower atmosphere and a fast enough computer with enough memory to cruch the data and accurately report what the weather will be like on 29 February.
    - doctor_eval 4y ago
      Are we thinking about the same femtometer? 10^-15 of a meter? https://en.wikipedia.org/wiki/Femtometre https://en.wikipedia.org/wiki/Femtometre I mean you can’t even fit a thermometer into a cubic femtometer..?
      - Certhas 4y ago
        I think Maursault has thoroughly demonstrated their lack of serious thought or reading on the subject. But just for giggles and for the casual reader, the lattice spacing of silicon is 200,000 femtometer. So if you encode only one bit per cube of this fm cubic lattice, and you manage to encode this into single atoms of silicon, you need a volume of silicon 8,000,000,000,000,000 times larger than the system you model.
        Maursault 4y ago
        > I think Maursault has thoroughly demonstrated their lack of serious thought When you can't beat the argument, pull out the ad hominem fallacy and attack the man. Fallacy, of course, is faulty reasoning. > So if you encode only one bit per cube of this fm cubic lattice, and you manage to encode this into single atoms of silicon, you need a volume of silicon 8,000,000,000,000,000 times larger than the system you model. This explanation is indicative of linear thinking. Apparently Google Earth is not possible, as it would require a computer the size of the planet. Digitizing the Library of Congress apparently requires a memory stick the size of Congress. Seriously? You just can not comprehend how things could ever get better than your current understanding of how things are right now? Consider that if you lived in 1500BC, were an expert at the time in farming, and a plough was described to you, you would mock the person describing it, and insist that tilling soil was impossible.
- adrianN 4y ago
  It's extremely unlikely that we'll ever get anywhere near that. Even meter precision is impractical.
  - Maursault 4y ago
    Unless you know it to be physically or logically impossible, you could not really know how likely it is or isn't. Ask anyone in the mid-1970's how likely it is that billions of people would be walking around with a supercomputer in their pockets, and they'd come up with all sorts of reasons why it was extremely unlikely, such as no individual would ever need so much computing power. The practicality of the precision only depends on the ability to measure and the ability to manipulate and simulate large amounts of data, both of which are extremely likely to get better, and better faster and faster, as time and technology progresses.
    - adrianN 4y ago
      A femtometer is a few orders of magnitude smaller than an atom. There are about 2*140 atoms in the atmosphere. You can't even count to that number, let alone do any fluid dynamics to that. I'm confident that we won't have femtometer scale simulations of the atmosphere before the sun becomes a red giant and swallows the earth.
      - Maursault 4y ago
        > A femtometer is a few orders of magnitude smaller than an atom. Thank you for telling me what a femtometer is, as though my using the word wasn't a pretty good indicator I knew what it was. You mean a femtometer is a real thing? And I just made that up out of thin air to mean a meter stick to give to women. What are the odds? > There are about 2*140 atoms in the atmosphere. You can't even count to that number, let alone do any fluid dynamics to that. Would you like me to explain how your argument is a straw man, or can I trust you to figure it out? > I'm confident that we won't have femtometer scale simulations of the atmosphere before the sun becomes a red giant and swallows the earth. Very colorful, but all you're really saying is that you are pessimistic about technology and about any staggeringly large advancements in computer design or weather sensor tech, while I, otoh, optimistically say I just don't know, but I bet computers will get faster, smaller and cheaper, and that within only a hundred years there will be weather tech that we are incapable of conceiving of today. Sure, a femtometer is mind-bogglingly small, but it's only 15 orders of magnitude smaller than a meter. It's way bigger than a zeptometer. How is it even possible femtometers can be described so simply? But of course, there could never be any more advancements in mathematics, physics, computer engineering or our current understanding of weather and climate. We basically know all there is to know right now. Huh.

Which is *exactly* why ultra-high resolution global weather simulation has dubious prospects for improving forecasts. When you're at spatial scales where you ne

Maursault 4y ago

If we're ever going to get to the femtometer resolution required for very precise 100 day weather forecasting, we have to start somewhere, so let them waste their time. It's not as though this is part of a growing trend to abandon conventional weather and climate modeling.

adgjlsfhk1 4y ago
there's a very good physical argument that this is impossible. if you want to store 1 bit per femptometer simulated, at current computer sizes, we are taking about a computer billions the size of the earth. even if you use 1 atom per bit, your computer will be almost as big as the earth. such a computer will collapse under it's own gravity.
- Maursault 4y ago
  > at current computer sizes This. No, not at all at current computer sizes, but at future computer sizes. This is the same mistake someone in the 1970's might make about billions having a smartphone today (supercomputer by their standards). Consider how everything at current computer sizes is effectively two dimensional, even stacked processors are still fundamentally 2D designs. There is still a lot of computing advancement ahead. 40 years from now they'll look back and think the same things we think when we look back 40 years, that the machines were so primitive, hardly anything could be done with them, and some will be nostalgic for them, talk about their strengths, while others will shake their heads and think even messing with the fastest workstation today is a waste of time. Just because we can't conceive of how, doesn't mean it's not possible, some day.
  - adgjlsfhk1 4y ago
    do the math at 1 atom per bit.
counters 4y ago
Why do you think that we need "femtometer resolution" for "very precise" 100 day weather forecasting? What even is "very precise" 100 day weather forecasting? I think it's very amusing to do the math on how much memory would be required to run a crude primitive equation dycore over even the tiniest of domains at femtometer resolution :) > It's not as though this is part of a growing trend to abandon conventional weather and climate modeling. The thing is, there *absolutely is* a trend towards private investment in weather modeling going towards faux-moonshot ideas like cubesat constellations without demonstrated ROI and that would require evolutionary leaps forward in data assimilation, or for deep learning to replace weather models. A miniature version of this already played out with precipitation nowcasting - probably the easiest weather forecasting problem that you could approach with an AI system, yet the approaches that have been developed so far barely improve over optical flow or other simple approaches, let alone advance our capability to forecast, say, convective initiation. The future of weather forecasting is larger ensembles (O(100-500) ensemble members, across 2-5 different models) of near-convective-resolving global models at meso-gamma (2-10 km resolution) fed into slightly more sophisticated statistical post-processing systems - almost certainly trained using simple AI/ML techniques on large-scale reforecasts of these parent model systems, or brute-forcing purely Bayesian statistical approaches.
- Maursault 4y ago
  > Why do you think that we need "femtometer resolution" for "very precise" 100 day weather forecasting? Due to sensitive dependence on initial conditons. Even using measurements at meter resolution will cause the accuracy of a forecast to begin to break down after only a few days. > What even is "very precise" 100 day weather forecasting? Anywhere from accurate to exact. > I think it's very amusing to do the math on how much memory would be required to run a crude primitive equation dycore over even the tiniest of domains at femtometer resolution And Bill Gates thought 64K should be enough for anybody. Do you really think computers will only have a few GB of memory 50 years from now? > there absolutely is a trend towards private investment in weather modeling going towards faux-moonshot ideas like cubesat constellations without demonstrated ROI and that would require evolutionary leaps forward in data assimilation, or for deep learning to replace weather models This straw man does not exactly demonstrate that conventional weather and climate modeling is being abandoned anytime soon. If the unconventional private investments aren't profitable, the market will deal with them. > The future of weather forecasting is much like the local weather, impossible to predict with any accuracy years into the future, and yet the tools used to measure it are consistently getting more accurate, cheaper and smaller. Maybe like bottle-openers, weather sensors may superfluously start appear on everything. The more widespread the measurements, the more data descibing initial conditions, the better the forecast will be at any interval.
contravariant 4y ago
You want to know the precise shape of the Earth's surface in femtometer precision? There are some profound problems with that idea once you get below 10 meter or so, but I'll let you think that one through yourself.
- Maursault 4y ago
  No, but I wouldn't mind weather measuments every cubic femtometer of the lower atmosphere and a fast enough computer with enough memory to cruch the data and accurately report what the weather will be like on 29 February.
  - doctor_eval 4y ago
    Are we thinking about the same femtometer? 10^-15 of a meter? https://en.wikipedia.org/wiki/Femtometre https://en.wikipedia.org/wiki/Femtometre I mean you can’t even fit a thermometer into a cubic femtometer..?
    - Certhas 4y ago
      I think Maursault has thoroughly demonstrated their lack of serious thought or reading on the subject. But just for giggles and for the casual reader, the lattice spacing of silicon is 200,000 femtometer. So if you encode only one bit per cube of this fm cubic lattice, and you manage to encode this into single atoms of silicon, you need a volume of silicon 8,000,000,000,000,000 times larger than the system you model.
      - Maursault 4y ago
        > I think Maursault has thoroughly demonstrated their lack of serious thought When you can't beat the argument, pull out the ad hominem fallacy and attack the man. Fallacy, of course, is faulty reasoning. > So if you encode only one bit per cube of this fm cubic lattice, and you manage to encode this into single atoms of silicon, you need a volume of silicon 8,000,000,000,000,000 times larger than the system you model. This explanation is indicative of linear thinking. Apparently Google Earth is not possible, as it would require a computer the size of the planet. Digitizing the Library of Congress apparently requires a memory stick the size of Congress. Seriously? You just can not comprehend how things could ever get better than your current understanding of how things are right now? Consider that if you lived in 1500BC, were an expert at the time in farming, and a plough was described to you, you would mock the person describing it, and insist that tilling soil was impossible.
adrianN 4y ago
It's extremely unlikely that we'll ever get anywhere near that. Even meter precision is impractical.
- Maursault 4y ago
  Unless you know it to be physically or logically impossible, you could not really know how likely it is or isn't. Ask anyone in the mid-1970's how likely it is that billions of people would be walking around with a supercomputer in their pockets, and they'd come up with all sorts of reasons why it was extremely unlikely, such as no individual would ever need so much computing power. The practicality of the precision only depends on the ability to measure and the ability to manipulate and simulate large amounts of data, both of which are extremely likely to get better, and better faster and faster, as time and technology progresses.
  - adrianN 4y ago
    A femtometer is a few orders of magnitude smaller than an atom. There are about 2*140 atoms in the atmosphere. You can't even count to that number, let alone do any fluid dynamics to that. I'm confident that we won't have femtometer scale simulations of the atmosphere before the sun becomes a red giant and swallows the earth.
    - Maursault 4y ago
      > A femtometer is a few orders of magnitude smaller than an atom. Thank you for telling me what a femtometer is, as though my using the word wasn't a pretty good indicator I knew what it was. You mean a femtometer is a real thing? And I just made that up out of thin air to mean a meter stick to give to women. What are the odds? > There are about 2*140 atoms in the atmosphere. You can't even count to that number, let alone do any fluid dynamics to that. Would you like me to explain how your argument is a straw man, or can I trust you to figure it out? > I'm confident that we won't have femtometer scale simulations of the atmosphere before the sun becomes a red giant and swallows the earth. Very colorful, but all you're really saying is that you are pessimistic about technology and about any staggeringly large advancements in computer design or weather sensor tech, while I, otoh, optimistically say I just don't know, but I bet computers will get faster, smaller and cheaper, and that within only a hundred years there will be weather tech that we are incapable of conceiving of today. Sure, a femtometer is mind-bogglingly small, but it's only 15 orders of magnitude smaller than a meter. It's way bigger than a zeptometer. How is it even possible femtometers can be described so simply? But of course, there could never be any more advancements in mathematics, physics, computer engineering or our current understanding of weather and climate. We basically know all there is to know right now. Huh.