5 ms·

This is a bit over-hyped and not exactly a breakthrough. It’s not doing true weather prediction but rather extrapolating the movement of radar images. This is n

by JCM9 5y ago

This is a bit over-hyped and not exactly a breakthrough. It’s not doing true weather prediction but rather extrapolating the movement of radar images. This is nothing new. I remember as far back as the 1990s TV weathermen would draw a line across a line of moving rain and the computer would predict which town it would get to and when. Slapping “AI” on this 25 years later is a good example of the whole present PR move of labeling things as “AI” that are just rather basic data analytics.

antegamisou 5y ago
Yep, unless they came up with a way to make the primitive equations solutions significantly computionally cheaper, you’re not getting any more accurate without better turbulence models.
netcan 5y ago
Possibly, but this is a DeepMind result. They've had enough non trivial successes to earn some credibility. Even if it is hype, more understandable than most overhyped PR headlines.
ironrabbit 5y ago
I don't know much about meteorology -- what is "true weather prediction" then?
- JCM9 5y ago
  The weather models that run on super computers and predict 3D detailed atmospheric conditions. This is just image analytics. These models could predict, for example, that new precip will develop. This just tracks the path of existing weather.
  - ironrabbit 5y ago
    It seems like this method outperforms those model-based approaches, why should we care that it's "just image analytics"?
    - counters 5y ago
      Because it means that the technique may have little to no utility for virtually any other weather forecasting application.
  - sdenton4 5y ago
    From the paper: "Ensemble numerical weather prediction (NWP) systems, which simulate coupled physical equations of the atmosphere to generate multiple realistic precipitation forecasts, are natural candidates for nowcasting as one can derive probabilistic forecasts and uncertainty estimates from the ensemble of future predictions7. For precipitation at zero to two hours lead time, NWPs tend to provide poor forecasts as this is less than the time needed for model spin-up and due to difficulties in non-Gaussian data assimilation8,9,10." Sounds like the detailed models are too heavy for this particular job, and that the existing methods to deal with it are too coarse. And there's lots of training data, so it's a really natural place to drop in a generative model. It's a special corner case of weather forecasting, but a real result.
  - jazzyjackson 5y ago
    Just think of the images as a PCA/dimensionality reduction. Instead of running a model on 3D atmospheric conditions, you run predictions based on a 2D slice of it. Enough of the information remains embedded in the radar images that predictions have similar accuracy to crunching numbers on higher dimensions.
FartyMcFarter 5y ago
> I remember as far back as the 1990s TV weathermen would draw a line across a line of moving rain and the computer would predict which town it would get to and when. Did it do it as accurately as this new method?
- JCM9 5y ago
  Those methods have improved too of course over the last few decades, but yes even back then it was quite accurate. It’s not terribly complicated. You take the image, see where it’s moving and at what speed and just forecast that into the future.
  - whimsicalism 5y ago
    Weather reporting has gotten notably much more accurate during my lifetime. You are making this sound way too simplistic - and sort of insulting to scientists who study weather patterns, it is not nearly as "draw a straight line and be right 100% of the time" as you're making it out to be, especially over larger timescales.
jhrmnn 5y ago
This is a very simplistic view. The complexity of weather prediction is a factor of both how far into the future and how fine-grained the prediction is. Predicting 1 minute ahead is trivial. Predicting temperature with a 20-degree accuracy a week ahead is trivial. What Deepmind does here is predicting precipitation ~1 hour ahead with a ~1km resolution, and they do it significantly better than existing models. Perhaps not a breakthrough, but a substantial technical development it is.
jasode 5y ago
>This is a bit over-hyped and not exactly a breakthrough. [...] This is nothing new. >Slapping “AI” on this 25 years later is a good example of the whole present PR move of labeling things as “AI” that are just rather basic data analytics. The New Atlas article has a link to the Nature journal paper it's based on. Your dismissal and summary of DeepMind's work described by the article as a "public relations move" is a disservice to readers. The more detailed explanation in the Nature journal describes a new technique of "deep generative models" applied to weather radar. This was not available 25 years ago. In tests, their DGM forecasts became preferred by meteorologists 93% of the time for accuracy compared to previous "data analytics". Excerpt from Nature: >We use a single case study to compare the nowcasting performance of the generative method DGMR to three strong baselines: PySTEPS, a widely used precipitation nowcasting system based on ensembles, considered to be state-of-the-art3,4,13; UNet, a popular deep learning method for nowcasting15; and an axial attention model, a radar-only implementation of MetNet19 >[...] When expert meteorologists judged these predictions against ground truth observations, they significantly preferred the generative nowcasts, with 93% of meteorologists choosing it as their first choice (Fig. 4b).
- JCM9 5y ago
  The underlying research is indeed very interesting. Passing this (image extrapolation as a means of tracking rain movements) off as something new or “AI” is what’s not great and doesn’t help trend of people rushing to portray something as “AI” when it’s just the same sort of data analytics that’s been going on for a long time. Granted a lot of that concern falls on the press and PR folks more so than the underlying research.
  - jasode 5y ago
    >Passing this (image extrapolation as a means of tracking rain movements) off as something new You're still misrepresenting the actual words in the NA article. NA didn't say "image extrapolation is new". What they actually wrote was the "generative modeling approach" was new. Excerpt: >DeepMind set out to develop a machine-learning tool that can bring a new level of precision to these efforts, [...] It did so by using a generative modeling approach, That's basically an accurate short summary of the longer Nature journal article. >or “AI” is what’s not great and doesn’t help trend of people rushing to portray something as “AI” when it’s just the same sort of data analytics This is really just the AI vs AGI[1] distinction. If you're not aware, the label "AI" (naked with no modifiers) has already been downgraded to "weak AI". So the more difficult "AI" that nobody solved yet is now elevated with a new label of "AGI" or "strong AI" to help sort out the confusion. - "AI" which implies "weak AI" : analogous to "we don't care that planes and drones don't really 'fly' because even if they don't flap their wings like birds, it still solves a problem." Analogy is "AI":"fly" --vs-- - "AGI" artificial general intelligence: solve the very hard problem of going from logic gates to "learning" everything like a child's brain does. In this definition, AlphaZero beating every human chess player and all previous computer chess engines is "not really AI". [1] https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...
andreyk 5y ago
I mean... You can look at the paper, they actually do use a novel generative AI model, so it's rather strange to criticize this being labeled AI. And, the paper also shows the new model to outperform existing ones in 84% of cases according to a bunch of human forecasters, so calling this just PR is overly cynical IMO. As far as "It’s not doing true weather prediction but rather extrapolating the movement of radar images.", both the paper and article say the paper is tackling short term rain prediction ('precipitation nowcasting'), so it's not oversold as far as I can see.
qwertox 5y ago
I think it was in an interview with someone from the ECMWF where they claimed that for each decade of progress, the accuracy of their models improved by one day. So we're currently at around three days of good forecasting. These models do not use AI, they work by extrapolating, like you say. If you have access to DWD's RADOLAN image data, for example as rendered images through the DWD WarnWetter-App (you need to pay a small one-time fee to access the radar data), you can clearly see how much this extrapolation leaves to be desired (even though it is extremely useful as it is). Actually, almost every German weather data provider which offers radar precipitation predictions is based on the raw data provided by DWD, this raw data can also be downloaded for free at https://opendata.dwd.de/weather/radar/radolan/rw/ https://opendata.dwd.de/weather/radar/radolan/rw/ Anyway, if you look at the predictions, they are pretty simple. As if the wind direction at two different altitudes is determined for each point, and then applied to the current precipitation data. These wind vectors don't change during the (short term, max 2h) prediction, so you see the parts of the image moving at a constant velocity as soon as you're talking about the future. This neglects two things: wind direction will change during these two hours, which is why you as the app user need to check often to verify if it is still accurate, but most importantly this simple model does not take into account the humidity in the air. So sometimes the rain will arrive sooner not because the wind got faster, but because new clouds are starting to build faster in your direction than the old cloud systems get to travel towards you with the wind. And in both these cases AI provides a significant potential of improvement. By looking at more of the surrounding weather dynamics it will be able to predict better what is actually happening in the weather system. Currently we can only improve this by adding more sensors and more frequent radar scans, but AI can really start to interpret the past one-hour-weather and "understand" what is happening there in order to predict what will happen later. And there is a ton of data available for training.
- counters 5y ago
  A quibble with wind direction and humidity. Really, what's needed is to address the kinematic/thermodynamic parameters more generally that might support (a) maintenance of the existing convection and (b) the propagation of new convective cells. The steering currents really don't change much over 2 hours for an organized system. You can get some rotational motion with a large cyclone but modern OF methods do just fine with that, and you can always remove the divergent component of the flow field. An example of (b) can be found in any Spring season convective outbreak in the Central US; once a squall line congeals along a front you'll see pioneer convection propagate along a vector somewhat orthogonal to the squall line's motion (there are heuristics for the propagation vector that work OK for curved hodographs except in inhomogeneous environments, e.g. Fig 8.10 from Markowski and Richardson). It's the 3D wind shear that matters here, augmented with the lapse rate / profile for (a). It's hard to bullish on the AI applications here until we see them start to account for these larger input parameter spaces. But of course, where is this data going to come from? Mesoscale or convection-permitting models. And if you already have the capability to run these models in a cost-efficient manner, do you need the AI system in the first place?
whimsicalism 5y ago
The top HN comments on anything related to ML are comically pessimistic and also extremely repetitive.

by JCM9 5y ago

antegamisou 5y ago

Yep, unless they came up with a way to make the primitive equations solutions significantly computionally cheaper, you’re not getting any more accurate without better turbulence models.

netcan 5y ago

Possibly, but this is a DeepMind result. They've had enough non trivial successes to earn some credibility. Even if it is hype, more understandable than most overhyped PR headlines.

ironrabbit 5y ago

I don't know much about meteorology -- what is "true weather prediction" then?

JCM9 5y ago
The weather models that run on super computers and predict 3D detailed atmospheric conditions. This is just image analytics. These models could predict, for example, that new precip will develop. This just tracks the path of existing weather.
- ironrabbit 5y ago
  It seems like this method outperforms those model-based approaches, why should we care that it's "just image analytics"?
  - counters 5y ago
    Because it means that the technique may have little to no utility for virtually any other weather forecasting application.
- sdenton4 5y ago
  From the paper: "Ensemble numerical weather prediction (NWP) systems, which simulate coupled physical equations of the atmosphere to generate multiple realistic precipitation forecasts, are natural candidates for nowcasting as one can derive probabilistic forecasts and uncertainty estimates from the ensemble of future predictions7. For precipitation at zero to two hours lead time, NWPs tend to provide poor forecasts as this is less than the time needed for model spin-up and due to difficulties in non-Gaussian data assimilation8,9,10." Sounds like the detailed models are too heavy for this particular job, and that the existing methods to deal with it are too coarse. And there's lots of training data, so it's a really natural place to drop in a generative model. It's a special corner case of weather forecasting, but a real result.
- jazzyjackson 5y ago
  Just think of the images as a PCA/dimensionality reduction. Instead of running a model on 3D atmospheric conditions, you run predictions based on a 2D slice of it. Enough of the information remains embedded in the radar images that predictions have similar accuracy to crunching numbers on higher dimensions.

FartyMcFarter 5y ago

> I remember as far back as the 1990s TV weathermen would draw a line across a line of moving rain and the computer would predict which town it would get to and when. Did it do it as accurately as this new method?

JCM9 5y ago
Those methods have improved too of course over the last few decades, but yes even back then it was quite accurate. It’s not terribly complicated. You take the image, see where it’s moving and at what speed and just forecast that into the future.
- whimsicalism 5y ago
  Weather reporting has gotten notably much more accurate during my lifetime. You are making this sound way too simplistic - and sort of insulting to scientists who study weather patterns, it is not nearly as "draw a straight line and be right 100% of the time" as you're making it out to be, especially over larger timescales.

jhrmnn 5y ago

This is a very simplistic view. The complexity of weather prediction is a factor of both how far into the future and how fine-grained the prediction is. Predicting 1 minute ahead is trivial. Predicting temperature with a 20-degree accuracy a week ahead is trivial. What Deepmind does here is predicting precipitation ~1 hour ahead with a ~1km resolution, and they do it significantly better than existing models. Perhaps not a breakthrough, but a substantial technical development it is.

jasode 5y ago

>This is a bit over-hyped and not exactly a breakthrough. [...] This is nothing new. >Slapping “AI” on this 25 years later is a good example of the whole present PR move of labeling things as “AI” that are just rather basic data analytics. The New Atlas article has a link to the Nature journal paper it's based on. Your dismissal and summary of DeepMind's work described by the article as a "public relations move" is a disservice to readers. The more detailed explanation in the Nature journal describes a new technique of "deep generative models" applied to weather radar. This was not available 25 years ago. In tests, their DGM forecasts became preferred by meteorologists 93% of the time for accuracy compared to previous "data analytics". Excerpt from Nature: >We use a single case study to compare the nowcasting performance of the generative method DGMR to three strong baselines: PySTEPS, a widely used precipitation nowcasting system based on ensembles, considered to be state-of-the-art3,4,13; UNet, a popular deep learning method for nowcasting15; and an axial attention model, a radar-only implementation of MetNet19 >[...] When expert meteorologists judged these predictions against ground truth observations, they significantly preferred the generative nowcasts, with 93% of meteorologists choosing it as their first choice (Fig. 4b).

JCM9 5y ago
The underlying research is indeed very interesting. Passing this (image extrapolation as a means of tracking rain movements) off as something new or “AI” is what’s not great and doesn’t help trend of people rushing to portray something as “AI” when it’s just the same sort of data analytics that’s been going on for a long time. Granted a lot of that concern falls on the press and PR folks more so than the underlying research.
- jasode 5y ago
  >Passing this (image extrapolation as a means of tracking rain movements) off as something new You're still misrepresenting the actual words in the NA article. NA didn't say "image extrapolation is new". What they actually wrote was the "generative modeling approach" was new. Excerpt: >DeepMind set out to develop a machine-learning tool that can bring a new level of precision to these efforts, [...] It did so by using a generative modeling approach, That's basically an accurate short summary of the longer Nature journal article. >or “AI” is what’s not great and doesn’t help trend of people rushing to portray something as “AI” when it’s just the same sort of data analytics This is really just the AI vs AGI[1] distinction. If you're not aware, the label "AI" (naked with no modifiers) has already been downgraded to "weak AI". So the more difficult "AI" that nobody solved yet is now elevated with a new label of "AGI" or "strong AI" to help sort out the confusion. - "AI" which implies "weak AI" : analogous to "we don't care that planes and drones don't really 'fly' because even if they don't flap their wings like birds, it still solves a problem." Analogy is "AI":"fly" --vs-- - "AGI" artificial general intelligence: solve the very hard problem of going from logic gates to "learning" everything like a child's brain does. In this definition, AlphaZero beating every human chess player and all previous computer chess engines is "not really AI". [1] https://en.wikipedia.org/wiki/Artificial_general_intelligence https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

andreyk 5y ago

I mean... You can look at the paper, they actually do use a novel generative AI model, so it's rather strange to criticize this being labeled AI. And, the paper also shows the new model to outperform existing ones in 84% of cases according to a bunch of human forecasters, so calling this just PR is overly cynical IMO. As far as "It’s not doing true weather prediction but rather extrapolating the movement of radar images.", both the paper and article say the paper is tackling short term rain prediction ('precipitation nowcasting'), so it's not oversold as far as I can see.

qwertox 5y ago

I think it was in an interview with someone from the ECMWF where they claimed that for each decade of progress, the accuracy of their models improved by one day. So we're currently at around three days of good forecasting. These models do not use AI, they work by extrapolating, like you say. If you have access to DWD's RADOLAN image data, for example as rendered images through the DWD WarnWetter-App (you need to pay a small one-time fee to access the radar data), you can clearly see how much this extrapolation leaves to be desired (even though it is extremely useful as it is). Actually, almost every German weather data provider which offers radar precipitation predictions is based on the raw data provided by DWD, this raw data can also be downloaded for free at https://opendata.dwd.de/weather/radar/radolan/rw/ https://opendata.dwd.de/weather/radar/radolan/rw/ Anyway, if you look at the predictions, they are pretty simple. As if the wind direction at two different altitudes is determined for each point, and then applied to the current precipitation data. These wind vectors don't change during the (short term, max 2h) prediction, so you see the parts of the image moving at a constant velocity as soon as you're talking about the future. This neglects two things: wind direction will change during these two hours, which is why you as the app user need to check often to verify if it is still accurate, but most importantly this simple model does not take into account the humidity in the air. So sometimes the rain will arrive sooner not because the wind got faster, but because new clouds are starting to build faster in your direction than the old cloud systems get to travel towards you with the wind. And in both these cases AI provides a significant potential of improvement. By looking at more of the surrounding weather dynamics it will be able to predict better what is actually happening in the weather system. Currently we can only improve this by adding more sensors and more frequent radar scans, but AI can really start to interpret the past one-hour-weather and "understand" what is happening there in order to predict what will happen later. And there is a ton of data available for training.

counters 5y ago
A quibble with wind direction and humidity. Really, what's needed is to address the kinematic/thermodynamic parameters more generally that might support (a) maintenance of the existing convection and (b) the propagation of new convective cells. The steering currents really don't change much over 2 hours for an organized system. You can get some rotational motion with a large cyclone but modern OF methods do just fine with that, and you can always remove the divergent component of the flow field. An example of (b) can be found in any Spring season convective outbreak in the Central US; once a squall line congeals along a front you'll see pioneer convection propagate along a vector somewhat orthogonal to the squall line's motion (there are heuristics for the propagation vector that work OK for curved hodographs except in inhomogeneous environments, e.g. Fig 8.10 from Markowski and Richardson). It's the 3D wind shear that matters here, augmented with the lapse rate / profile for (a). It's hard to bullish on the AI applications here until we see them start to account for these larger input parameter spaces. But of course, where is this data going to come from? Mesoscale or convection-permitting models. And if you already have the capability to run these models in a cost-efficient manner, do you need the AI system in the first place?

whimsicalism 5y ago

The top HN comments on anything related to ML are comically pessimistic and also extremely repetitive.