counters

14 ms·

151.
by counters 3y ago
My understanding is that they just use an AI-based precipitation nowcast (see [1]). Very different forecast/modeling problem than GraphCast. [1]: https://arxiv.org/abs/1905.09932
152.
by counters 3y ago
Pragmatically speaking, it doesn't really matter if one is better than the other, at least until there is a massive jump in forecast quality (e.g. advancing the Day 5 accuracy up to Day 3). In the real world, we would never take raw mo
153.
by counters 3y ago
At that point you're slipping into Laplace's Demon. In practical terms, we see predictability horizons get _shorter_ when we increase observation density and spatial resolution of our models, because more, small errors from slight
154.
by counters 3y ago
Weather models are routinely run at resolutions as fine as 1-3 km - fine enough that we do not parameterize things like convection and allow the model to resolve these motions on its native grid. We typically do this over limited areas (e.g
155.
by counters 3y ago
GraphCast does predict rainfall - see https://charts.ecmwf.int/products/graphcast_medium-rain-acc?... for example.
156.
by counters 3y ago
Yes, this is effectively what 4DVar data assimilation is [1]. But it's very, very expensive to continually run new forecasts with re-assimilated state estimates. Actually, one of the _biggest_ impacts that models like GraphCast might h
157.
by counters 3y ago
They're more-or-less the same thing.
158.
by counters 3y ago
What you're describing is effectively how climate models work; we run a physical model which solves the equations that govern how the atmosphere works out forward in time for very long time integrations. You get "daily weather&quo
159.
by counters 3y ago
You can try, but other models in this class have struggled when initialized using model states pulled from other analysis systems. ECMWF publishes a tool that can help bootstrap simple inference runs with different AI models [1] (they have
160.
by counters 3y ago
Well, what is "real weather data?" We have dozens of complementary and contradictory sources of weather information. Different types of satellites measuring EM radiation in different bands, weather stations, terrestrial weather ra
161.
by counters 3y ago
Absolutely - but large ensembles are just the tip of the iceberg. Why bother producing an ensemble when you could just output the posterior distribution of many forecast predictands on a dense grid? One could generate the entire ensemble-de
162.
by counters 3y ago
It's worth pointing out that "state of the weather" is a little bit hand-wavy. The GraphCast model requires a fully-assimilated 3D atmospheric state - which means you still need to run a full-complexity numerical weather pred
163.
by counters 3y ago
It doesn't seem like MetNet outputs a complete 3D atmospheric state, just specific (and mostly surface-level) forecast predictands [1]. The analysis you're describing could be done with a traditional numerical weather prediction a
164.
by counters 3y ago
Windy visualizations raw or direct outputs from the weather forecast models run by agencies in the USA and Europe. These models are the "bread and butter" of operational weather forecasting, and are the baseline in accuracy for al
165.
by counters 3y ago
In practice this is happening in many disciplines, for most research, on a daily basis. What _isn't_ happening is that the results of these replications are being independently peer reviewed, because that isn't incentivized. Howev
166.
by counters 3y ago
The problem in this world of AI/weather is that "accuracy" is an extremely fuzzy concept. The leading pack of AI-NWP models (NVIDIA's FourCastNet, DeepMind's Graphcast, Huawei PanguWeather), when compared on an appl
167.
by counters 3y ago
> I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry... The problem is that they offer little to no advantage over the highly optimized ML-based forecast post-pr
168.
by counters 3y ago
ML/AI has been used in the weather forecasting world since the 1970's - most often to post-process forecast model output to better calibrate it against observations and correct for biases. > One would think this would be one of
169.
by counters 3y ago
It's specifically referring to these forecast models having accuracy about on par with the state-of-the-science global numerical weather forecast models. "Accuracy" here specifically means esoteric metrics like the "500m
170.
by counters 3y ago
lol I've suffered under that before... except it was an ex-Googler forcing bazel on us, all of a 10 person dev team working on a codebase that was probably less than 15,000 LOC across three or four packages that just _had_ to be packag
171.
by counters 3y ago
> Ultimately, why not simply compute it and give the user a warning that the local measurement isn't compatible with proper AQI measurement? You totally could, but in practice it doesn't always or obviously lead to a positive u
172.
by counters 3y ago
Probably the two most common and simplest are Barnes and Cressman interpolation (see [1] for a modern implementation of Barnes), which use inverse distance weighting to combine observations within a neighborhood. An improvement to these tec
173.
by counters 3y ago
Well, the underlying data might not actually be the same. One country may lump in PM10 into their AQI, so if there is a dust storm or similar event, the AQI could spike. Other countries may only include PM2.5 or not include PM at all, which
174.
by counters 3y ago
The problem here is twofold. First, many countries have their own laws and policies regarding air quality exposure. Often times these regulations use metrics about time-average exposure (e.g. the annual average 8-hr maximum ozone concentrat
175.
by counters 3y ago
Eh, kriging is overkill relative to the skill you end up with since fine-grain spatial variability in weather can arise from a lot more than the static factors you can pull into such a scheme. This is why objective analysis techniques are t
176.
by counters 3y ago
Hey, all the thanks go to you! I really wish we could do more to reward contributions like this when they're coming from folks in the academic world... the impact of this work is easily on par with anything else you could do as a junio
177.
by counters 3y ago
Yeah, I agree 100% - I really support the approach of the developer here and am totally aligned with all the reasoning. The libprima/prima codebase is very readable, even if you're not accustomed to modern Fortran (let alone Fortr
178.
by counters 4y ago
No, all you need to do is memorize enough vocabulary and have enough chances to get lucky on occasion. You can have better odds if you learn the conditional probability of that vocabulary given surrounding words. And this is all that GPT is
179.
by counters 4y ago
I don't think so. In practice, you'll need to hire _both_ the domain expert and the ML specialist. Or maybe even no change at all... you still want the domain expert, because the problems may be fundamentally related to the framin
180.
by counters 4y ago
> So explain that to me. You're special. Congratulations! You have a unique capability to absorb knowledge. In my experience, this means one of two things: either (1) you truly are brilliant and will exceed at anything you apply to,

More ›