36 ms·

The unnecessary decline of U.S. numerical weather prediction

205 points by carabiner 2y ago

ttyprintk 2y ago
This is a valuable blog on the subject, and his answer explains how the US could rectify. But NWS didn’t correct their hurricane forecast to the whims of a mercurial President. Talking about the technicalities of the model library is fine but only after you pass the loyalty test.
- jmclnx 2y ago
  Also I heard many smart people in NOAA left between 2016 and 2020 to go private and funding cuts. I would not be surprise many in NOAA are looking to leave now. I wonder if the listeria issues occurring now is also a symptom of this plus the FDA funding cuts over the past 20+ years.
  - clumsysmurf 2y ago
    Project 2025 describes NOAA as a primary component “of the climate change alarm industry” and said it “should be broken up and downsized.” Talent is probably worried about the future political climate as well. https://static.project2025.org/2025_MandateForLeadership_FULL.pdf#page=707 https://static.project2025.org/2025_MandateForLeadership_FUL...
    - cratermoon 2y ago
      It's will known that AccuWeather's president Joel Myers and his younger brother (now former) AccuWeather CEO Barry Lee Myers want to kill public domain weather report data. https://en.wikipedia.org/wiki/AccuWeather#National_Weather_Service https://en.wikipedia.org/wiki/AccuWeather#National_Weather_S... Barry was nominated (but never confirmated) for Under Secretary for Oceans and Atmosphere and Administrator, National Oceanic and Atmospheric Administration While AccuWeather denies involvement in Project 2025, there's definitely overlap between TFG's administration and the authors of the plan.
      - throw0101d 2y ago
        > It's will known that AccuWeather's president Joel Myers and his younger brother (now former) AccuWeather CEO Barry Lee Myers want to kill public domain weather report data. John Oliver of all people had a segment on this: * https://www.youtube.com/watch?v=qMGn9T37eR8 https://www.youtube.com/watch?v=qMGn9T37eR8 * https://www.imdb.com/title/tt11110660/ https://www.imdb.com/title/tt11110660/ * https://www.theguardian.com/culture/2019/oct/14/john-oliver-last-week-tonight-recap-national-weather-service https://www.theguardian.com/culture/2019/oct/14/john-oliver-... * https://time.com/5699545/john-oliver-weather-last-week-tonight/ https://time.com/5699545/john-oliver-weather-last-week-tonig...
  - counters 2y ago
    I haven't heard anything about NOAA having any new issues with workforce retention. The trend you're seeing in private sector expansion / growth is linked to the massive investments in Earth observation and derivative applications. It's not really a response to anything happening in government. If anything, the federal government forced some of this growth through commitments like the Commercial Weather Data Program.
  - ironhaven 2y ago
    You can draw a strong connection from in 2019[1] when the USDA allowed slaughterhouses to self inspect to the recent listeria outbreaks. [1] https://www.npr.org/sections/thesalt/2019/09/17/761682926/usda-changes-rules-overseeing-how-pork-is-produced https://www.npr.org/sections/thesalt/2019/09/17/761682926/us...
    - shiroiushi 2y ago
      It would be really interesting to interview all the listeria victims and find out how they voted. Elections have consequences.
  - renewiltord 2y ago
    Yes, indeed, the more they start sucking at things, the more money we should give them. This will incentivize not sucking.
    - Apocryphon 2y ago
      Hello, the spirit of No Child Left Behind
deleted 2y ago
[deleted]
mturmon 2y ago
Red hat at the end positions this as a job application?
- lithos 2y ago
  More like declaring the real plan of setting up one org that needs to be privatized. There have been real attempts at privatization of weather services in the US, despite the fact that our current system costs pennies a day per person (whether it's used for weather apps, or safe flying/sailing).
  - momoschili 2y ago
    unlikely that there is good precedent for a private weather organization. The gold standard for western weather prediction is the European Centre for Medium-Range Weather Forecasts and that is a collaboration between multiple government agencies.
    - mrguyorama 2y ago
      The "lets privatize essential parts of the government" folks do not care about precedent, or what is a good idea. They are driven by ideology and self dealing.
      - momoschili 2y ago
        even Cliff himself (somewhat right leaning in today's American political environment) did not go so far in his blog. Cliff often speaks very highly of the European weather forecasts, and in his blog post it seems pretty clear that he is using it as the model for most of the suggestions.
        throwway120385 2y ago
        The problem is that his well-meaning statements are going to be used as ammo to privatize NOAA, much like someone's well-meaning statements were used to create turmoil in Springfield, Missouri.
mnky9800n 2y ago
I just applied for a job at noaa after being asked by a director to apply for the job which was directly related to updating the core weather modelling libraries, some of which are 50+ years old Fortran, and also develop projects to integrate modern ai/ml into the modelling pipeline. I have 10+ years of experience working both as a researcher and a developer, a PhD in physics, and have won multiple grants either applying machine learning to geophysics problems or develop Bayesian spatiotemporal modeling libraries. At least on paper, I should get an interview for this job. I was rejected because: > We are not able to determine your qualifications as your resume does not show complete information for each job entry, such as beginning and ending dates of employment, duties performed, and/or total hours worked per week First, my cv did include everything but the total hours worked per week. Second there was no instruction that I could find for writing my CV such that I needed to list the number of hours worked per week. Third there was no way to contest this. Even the person who invited me to apply who would be the hiring manager had no control over this. I can tell you, this assuredly leads to the unnecessary decline of any system when there’s an impenetrable hr system sitting between applicants and jobs.
- deleted 2y ago
  [deleted]
- brudgers 2y ago
  Because NOAA is part of the Federal government, expect job applications to be inspected technically detailed relative to compliance with a mountain of Federal statutes and regulations. Because people who don't get hired can literally get their Congressional and Senate representatives involved. This makes getting hired into the Civil Service hard. Internal candidates are usually at an advantage. Veterans always are.
  - mnky9800n 2y ago
    I’m currently a civil servant of Norway. They seem to have it figured out. Haha. But yeah I get you.
  - warner25 2y ago
    > Internal candidates are usually at an advantage. Veterans always are. The system is as you described. Internal candidates already know it and veterans, when transitioning out of uniformed service, get a full-blown class on it: how to prepare their "Federal resume" (which is unlike any resume that a reasonable person might otherwise prepare) and navigate the hiring process. So it's that on top of already having relevant experience and credentials or preference points. I've been on the hiring side for Federal GS positions, and I'll tell you that it can be just as frustrating on that side too.
sampo 2y ago
> I have written two peer-reviewed papers in a major meteorological journal (here) The blogger is Cliff Mass, a professor at University of Washington. He has written two papers on this specific topic (decline of US weather predictions). In general, as a scientist, he has published 100+ papers. He only linked to his second paper [1]. I also found his first paper [2]. [1] "The Uncoordinated Giant II" (2023) https://journals.ametsoc.org/downloadpdf/view/journals/bams/104/4/BAMS-D-22-0037.1.pdf https://journals.ametsoc.org/downloadpdf/view/journals/bams/... [2] "The Uncoordinated Giant" (2006) https://www.e-education.psu.edu/files/meteo410/image/Lesson3/cliffmassBAMS0205.pdf https://www.e-education.psu.edu/files/meteo410/image/Lesson3...
- deleted 2y ago
  [deleted]
blueelephanttea 2y ago
I don't think the title of the article matches the content. The GFS has not declined relative to the ECMWF. It just started from a lower skill level and its improvements have not significantly closed the gap with ECMWF (as it has also improved). > Specifically, NOAA's global model, the UFS, is now in third or fourth place behind the European Center, the UK Meteorology Office, and often the Canadians. Would love to see evidence of that. It is well established that ECMWF is top in the game. I don't think it is reasonable to just state that Canada and UK are better without evidence. With that said, I agree the US should improve.
- marcyb5st 2y ago
  When it comes to medium range predictions. If you look at nowcasting, I believe the top dog is metnet 3 (https://arxiv.org/abs/2306.06079 https://arxiv.org/abs/2306.06079). On a tangential note, I worked on weather forecasting a bit before, and talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models. Not sure if they can't attract talent and so are somewhat stuck or if their higher ups are "old school" and can't really see the potential that this approach has to offer. It is just sad that the private sector is outclassing institutions and that these better models won't hit the public domain.
  - plantain 2y ago
    > talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models This is trivially verifiable as not true. ECMWF is all over AI: https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-forecasting-system https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-... https://github.com/ecmwf-lab/ai-models https://github.com/ecmwf-lab/ai-models And they are hiring with competitive salaries (for Europe).
    - AStonesThrow 2y ago
      If only we'd put forecasting on the blonkchain so we could defend against 51% tornadoes
MrMcCall 2y ago
[flagged]
- treyd 2y ago
  This is also the same process that the Conservative parties in the UK ans Canada have been carrying out with their respective healthcare systems, much to the suffering of their citizens.
  - MrMcCall 2y ago
    Indeed. One cannot get absurdly wealthy without making money from millions and millions of others. It's the nature of such wealth, and it is a sign of a society's moral decay. I'm not against making lots of money by providing a good product at a fair price, but that's not what right-wingers are doing. They are robber-barons robbing from the populace for their own selfish greed. And we are all left poorer, both literally and figuratively. Once we shift our personal and governmental focus to raising the standard of living for our least cared-for -- instead of allowing our systems to add more riches to the wealthy -- we can change this world into a fun, beautiful, and healthy place for us ALL. But that will not happen without selfless compassion. Selfish taking harms us all; selfless giving helps us all. To love someone is to want them to be happy, and we are in desperate need of far more lovingkindness in this world.
etiennebausson 2y ago
>> U.S. numerical weather prediction (NWP), which uses computer simulation to predict future weather, should be the best in the world. The author makes an important point, on a subject that keep getting more important due to climate change, but appeal to blind nationalism doesn't work in his favor. It's not so much a bias a willful blindness at this point. I have to tolerate it coming from politicians, but it is a different story when the author present himself as a serious scientist. Yes, the U.S. could be the best at meteorology. they could be the best at anything, with their means. But not at everything. The U.S. are the best at military spending, and everything else is at best in second place.
- rossdavidh 2y ago
  Well historically the military has quite often been one of the primary customers of weather forecasting, so that doesn't necessarily mean we couldn't have the best meteorology (assuming for the sake of argument that we don't).
  - eximius 2y ago
    It doesn't seem likely that the military has a private model that is dramatically better. If we did, it would likely be exposed in a way similar to GPS.
    - itishappy 2y ago
      They'll release an intentionally kneecapped model until it's clear that workarounds are well known enough to allow release of the actual data? https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_Positioning_System#Selective_Availability https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_...
jeffbee 2y ago
Interesting call for more horsepower behind numerical methods. One of the benefits of the emerging machine learning forecast models is dramatically lower need for computation (and therefore energy).
- counters 2y ago
  Sort of. The motivation here is that numerical weather prediction (NWP) models are used for a lot more than just weather forecasting - they're critical research infrastructure and tools. NWP allows researchers to simulate complex atmospheric flows and phenomena which might not be readily or directly observed - or to manipulate observed flows in ways to evaluate dynamical theory and the underlying physics. You can't really do any of this with the AI emulators, which output a tiny, tiny sliver of information relative to the vast array of data you can dump from an NWP system. Regarding energy usage, you're right up to a point. Today's ML weather models are trained to emulate very large reanalysis datasets (e.g., ECMWF's ERA5) which are produced using the physics-based models. No one has yet to demonstrate an end-to-end AI/ML weather forecast which sidesteps these reanalysis datasets (in fact, the trend so far has been to include _more_ physics-based model datasets for fine-tuning, including archives of historical NWP forecasts or GCM simulations). So there's a massive sunk cost in energy usage to create those training datasets, and then there's the ongoing energy cost of training AI emulators from the ground up. And of course, there's the future energy cost of running more physics-based models to better support the development of AI emulators. For pure inference/forecasting? Sure. Much faster to run AI models and much lower energy usage. But that's quite literally the tip of the iceberg when it comes to forecast model development.
  - jeffbee 2y ago
    Sure, the AI stuff is not a complete solution because it relies on the results of numerical methods. I was only suggesting that in the question of making up the marginal difference between the best and fourth-best weather predictions, there might be some application of these other novel methods. If the question is can the Americans build a gigantic supercomputer and write fortran programs, the answer to that seems obvious. Yes we can do that when the relevant bureaucracy gets pointed in the right direction.
throw0101d 2y ago
Somewhat related, but I found Andrew Blum's book The Weather Machine: A Journey Inside the Forecast an interesting read: > In The Weather Machine, Andrew Blum takes readers on a fascinating journey through an everyday miracle. In a quest to understand how the forecast works, he visits old weather stations and watches new satellites blast off. He follows the dogged efforts of scientists to create a supercomputer model of the atmosphere and traces the surprising history of the algorithms that power their work. He discovers that we have quietly entered a golden age of meteorology—our tools allow us to predict weather more accurately than ever, and yet we haven’t learned to trust them, nor can we guarantee the fragile international alliances that allow our modern weather machine to exist. * https://www.goodreads.com/book/show/42079139-the-weather-machine https://www.goodreads.com/book/show/42079139-the-weather-mac... * https://www.andrewblum.net/the-weather-machine-2 https://www.andrewblum.net/the-weather-machine-2 Also his book Tubes, the Internet's physical infrastructure. He also appears to be working on a book on the electrical grid: * https://www.andrewblum.net/electric-earth https://www.andrewblum.net/electric-earth
- warner25 2y ago
  Thanks for the recommendation. I read Tubes back in 2012 when it was first published, and I enjoyed it. I knew a lot less about networking back then, so it would probably be a good book to re-read now with a different perspective, and to think about how things have might have changed since then.
  - ForOldHack 2y ago
    Read Tubes last year. Magnificent read. A must read for internet users.
- therein 2y ago
  Aren't all weather forecasts getting their data from Raytheon in the US via Weather Central, LP?
M_bara 2y ago
Missed a chance to reword that epic as a failure to an epic failure
_yb2s 2y ago
As a sailor I’m really into getting a good weather prediction and most of the models are quite inaccurate. I learned on here about an ML based weather model startup that posted predictions on their website called Atmo.AI (edited) that was mind blowingly accurate and had incredible spacial resolution- it could accurately predict, e.g. the wind shadow behind islands, something none of the mainstream weather models even come close to doing. I used it everyday and it was almost always dead right at predicting wind speeds in almost any specific spot. I won a lot of sailboat races with it. I hope this tech gets taken up by NOAA, or other major weather predictors. Anyone on here know what happened to Atmo and if the tech is ever going to come back so people can use it? It's a travesty that this tech exists but isn't being used- a lot of lives would be saved if NOAA were using this already existing technology. Edit: Apparently as another poster pointed out I had just forgotten the correct name of it, and it is still around. I edited this to reflect that.
- carabiner 2y ago
  Do you mean atmo.ai? Atmos.ai seems to be something very different: "Atmos AI streamlines the Carbon Accounting, GHG Emissions, and Environmental, Social, and Governance (ESG) reporting processes for companies of all shapes and sizes."
  - _yb2s 2y ago
    Ok, I am embarrassed, I simply forgot the name of it, and thought it had disappeared when I tried to find it again. Seems it is still alive and well. I swear this isn't viral marketing for them, I was legitimately confused. I edited my post above to use the correct name, but realize it doesn't make too much sense anymore.
- code_biologist 2y ago
  How do I use Atmo for locations other than the bay area demo linked on their website? I tried URL tweaking without much luck. I have a friend who does a lot of sailing races who would love this!
  - 2y ago
jwr 2y ago
As a European, I can tell you not to worry too much. The EU forecasts are mostly garbage. For some reason (possibly climate change) weather forecasting became unreliable over the last 10-15 years. I regularly check 5 different forecasting models (through a subscription in the Windy app and elsewhere) and they are unable to agree on whether water will fall on my head later today. We're not talking long-range forecasting here, just tell me if it's going to rain in several hours or not. There is no forecasting model that gets this right.
- magicalhippo 2y ago
  Norwegian official weather forecasting service usually say they're quite accurate, and I was thinking how can that be? But in an interview last year they spilled the beans, they were talking about temperature and only missing by 1-2 degrees C most of the time. However, I don't care if it's 22C or 20C, like you I care about am I gonna need a jacket or not. I get that it's a difficult problem. And the inclusion of local weather radar to track actual rain has been very nice. But it only forecasts 90 minutes ahead, and rain can develop closer than that some times.
AIFounder 2y ago
[dead]
jccalhoun 2y ago
The article leaves out that members of one political party have proposed eliminating the NOAA entirely https://www.washingtonpost.com/weather/2024/07/22/project-2025-weather-service-trump/ https://www.washingtonpost.com/weather/2024/07/22/project-20...
- yogurtboy 2y ago
  Lol I shouldn't have clicked on that link, that shit really makes my blood boil. I don't think most people realize how much free value they get out of NOAA weather predictions.
  - nickff 2y ago
    While I’d agree that accurate weather forecasts provide tremendous value, it’s not “free value”, as tax dollars are paying for it. That money could go to any number of ‘under-funded’ other causes, or be returned to the tax-payers.
    - aegypti 2y ago
      A relief to know NOAA isn’t doing this pro bono, I was beginning to wonder whether they were simply donating forecasts.
- Spooky23 2y ago
  This stuff has been a big bugaboo for GOP idiots for a long time. Rick Santorum (a former conservative luminary) was pushing for a law that would prohibit making weather data available to the public, but would make it available to companies like Accuweather for free. The brother of the Accuweather founder was Trump’s appointee to run the weather service. It’s on of those things that illustrates that no public resource is safe from graft.
- tzs 2y ago
  They also want to get rid of most civil service employees and change their positions to political appointees. Not just the top management positions, but also the people that actually do the day to day work. E.g., the scientists at the FDA who check the science on safety and effectiveness when processing approvals for new drugs would be political appointees who were vetted for loyalty to the President.
CapeTheory 2y ago
Don't worry, UK Met Office are currently shitting the bed on a migration from on-prem HPC to Azure (3+ years late at this point I believe?) so the competition isn't exactly hot.
joe_the_user 2y ago
Specifically, NOAA's global model, the UFS, is now in third or fourth place behind the European Center, the UK Meteorology Office, and often the Canadians. So there's been no decline. In fact US weather prediction is getting better (by the article own chart). It's just The author is upset the US isn't moving ahead as quickly as other weather services. Maybe they should be upset by this but the way it's framed seems a bit dishonest.
looping__lui 2y ago
One of the fee startups I am genuinely impressed: skysight. https://magazine.weglide.org/skysight-interview-matthew-scutter-gliding/ https://magazine.weglide.org/skysight-interview-matthew-scut... Freakingly accurate weather forecasts for the niche sport of gliding. Builds on top of ECMWF and incorporates refined soil moisture. It’s really good in predicting mountain waves (and used by Perlan: https://perlanproject.org/perlans-sponsors-enable-us-to-fly-high/ https://perlanproject.org/perlans-sponsors-enable-us-to-fly-... ). Also convergences and thermal activity which is very dynamic… Incredibly impressive.
timthorn 2y ago
> Our nation invented the technology... Lewis Fry Richardson might have something to say about that.
GarnetFloride 2y ago
I am an amateur radio operator and looking into the history of it, you quickly run into the telegraph, and telegraph operators would talk about things like the weather. It didn't take long before they realized that the weather in a place 100 miles away could show up the next day. Then they would warn everyone around them if bad weather was coming.
lisper 2y ago
I have lived in California for 36 years, 22 in SoCal and 14 in NorCal and I've noticed a trend that I find deeply disturbing: weather forecasts seem to be much less reliable now than they used to be. Before moving to California I lived on the east coast and the weather forecasts were generally unreliable, so I was really struck by how predictable the weather in California was, and how accurate the forecasts were. Rain began and ended when it was predicted to within a couple of hours. The temperature was within a few degrees of what it was predicted to be. And the window of predictability went out 5+ days. Nowadays that has all changed. Forecast rain fails to materialize. Heat waves come out of nowhere and last much longer and are much more intense than they are forecast to be. Temperatures at my house regularly go 10 degrees above what is predicted, and regularly hit temperatures that would have been extremely anomalous when we first moved here. This is all anecdotal. I haven't been keeping records, and I haven't done the math. It's possible that this is all confirmation bias or bad memory at work. But damn, it sure seems like a real phenomenon, and the most likely explanation is that the models were trained on data that don't reflect current conditions because of climate change. The most striking thing about this is the speed with which things are changing. 36 years is nothing on what should be geologic time scales. I shudder to think what the future holds.
- rapjr9 2y ago
  I've seen something similar happen and think I figured out the reason. Weather forecasts in my area were very accurate for decades, then they got much worse. They were off in timing for events, and incorrect about whether rain was going to fall or not. Then I realized that the change happened around the time the local nuclear power plant was shut down. The plant had been funding a local weather station and weather radio station as part of government requirements to provide early warning of problems to citizens. When the plant shut down the weather station went with it. So NOAA was no longer getting accurate data from this area and the forecasts were off because of that. I've been thinking of setting up my own station and linking it into the public data to see if that makes forecasts get more accurate.
snakeyjake 2y ago
>In most nations, numerical weather prediction is the responsibility of one group. In the U.S., governmental global weather prediction is spread over FIVE efforts: >NOAA, the National Oceanographic and Atmospheric Administration (the leading U.S. effort) >The US Navy >The US Air Force >NASA >U.S. Department of Energy This doesn't make sense. Other nations don't have a NASA, and if they do have a space agency that space agency is almost certainly in the weather business. JAXA: https://earth.jaxa.jp/en/data/2547/index.html https://earth.jaxa.jp/en/data/2547/index.html They also don't have expeditionary militaries like the USAF and USN, which need weather for the middle of the ocean or remote nation, far from NOAA's ability to deploy weather balloons. And as far as I can tell, the only exposure to weather that the DOE has is for climate change reasons, and their interest in that should be obvious. There's no waste. NOAA does the US, DOD does DOD, NASA helps out, and the DOE is looking at how climate change will impact the US's energy security. So reason number 1 is horseshit, and I stopped reading.
8bitsrule 2y ago
Mass/Maas uses European results in his blog often, often commenting that the European predictions for the PNW are more accurate. So, fine, that's great. Why does the US have to keep up? Will it stop the decline of the Empire? Is it a stab in the heart of our world posture? Are we doomed to having to be the best at everything for all of eternity? I don't get it. I'm more upset about all the interest being paid on the US national debt that -could- have been spent on something great instead.
paulkon 2y ago
“In the U.S., governmental global weather prediction is spread over FIVE efforts: NOAA The US Navy The US Air Force NASA U.S. Department of Energy Thus, scientific and technical talent and computer resources are diffused over five groups, greatly undermining progress.“ I suppose if Elon has his way, the overlapping functions from these are on the chopping block.
dramm 2y ago
Run your own code :-) I used to fly sailplanes and around Northern California at least there is an interesting intersection of computer nerds, aerodynamics nerds and weather nerds. One of my favorite things being Jack Glendening's weather code/models for sailplane pilots, culminating in the RASP forecasts that more technical sailplane (and hang glider etc) pilots ran for their own hyper-local forecasts. The localized terrain triggered thermal hotspots and convergence prediction of these RASP models were super impressive. http://www.drjack.info/rasp http://www.drjack.info/rasp. Today specialized commercial forecast services are used by many sailplane pilots, and RASP and related forecasts predated or inspired those, but a well run RASP forecast on a fine grid is still something to behold.

205 points by carabiner 2y ago

This is a valuable blog on the subject, and his answer explains how the US could rectify. But NWS didn’t correct their hurricane forecast to the whims of a mercurial President. Talking about the technicalities of the model library is fine but only after you pass the loyalty test.

jmclnx 2y ago
Also I heard many smart people in NOAA left between 2016 and 2020 to go private and funding cuts. I would not be surprise many in NOAA are looking to leave now. I wonder if the listeria issues occurring now is also a symptom of this plus the FDA funding cuts over the past 20+ years.
- clumsysmurf 2y ago
  Project 2025 describes NOAA as a primary component “of the climate change alarm industry” and said it “should be broken up and downsized.” Talent is probably worried about the future political climate as well. https://static.project2025.org/2025_MandateForLeadership_FULL.pdf#page=707 https://static.project2025.org/2025_MandateForLeadership_FUL...
  - cratermoon 2y ago
    It's will known that AccuWeather's president Joel Myers and his younger brother (now former) AccuWeather CEO Barry Lee Myers want to kill public domain weather report data. https://en.wikipedia.org/wiki/AccuWeather#National_Weather_Service https://en.wikipedia.org/wiki/AccuWeather#National_Weather_S... Barry was nominated (but never confirmated) for Under Secretary for Oceans and Atmosphere and Administrator, National Oceanic and Atmospheric Administration While AccuWeather denies involvement in Project 2025, there's definitely overlap between TFG's administration and the authors of the plan.
    - throw0101d 2y ago
      > It's will known that AccuWeather's president Joel Myers and his younger brother (now former) AccuWeather CEO Barry Lee Myers want to kill public domain weather report data. John Oliver of all people had a segment on this: * https://www.youtube.com/watch?v=qMGn9T37eR8 https://www.youtube.com/watch?v=qMGn9T37eR8 * https://www.imdb.com/title/tt11110660/ https://www.imdb.com/title/tt11110660/ * https://www.theguardian.com/culture/2019/oct/14/john-oliver-last-week-tonight-recap-national-weather-service https://www.theguardian.com/culture/2019/oct/14/john-oliver-... * https://time.com/5699545/john-oliver-weather-last-week-tonight/ https://time.com/5699545/john-oliver-weather-last-week-tonig...
- counters 2y ago
  I haven't heard anything about NOAA having any new issues with workforce retention. The trend you're seeing in private sector expansion / growth is linked to the massive investments in Earth observation and derivative applications. It's not really a response to anything happening in government. If anything, the federal government forced some of this growth through commitments like the Commercial Weather Data Program.
- ironhaven 2y ago
  You can draw a strong connection from in 2019[1] when the USDA allowed slaughterhouses to self inspect to the recent listeria outbreaks. [1] https://www.npr.org/sections/thesalt/2019/09/17/761682926/usda-changes-rules-overseeing-how-pork-is-produced https://www.npr.org/sections/thesalt/2019/09/17/761682926/us...
  - shiroiushi 2y ago
    It would be really interesting to interview all the listeria victims and find out how they voted. Elections have consequences.
- renewiltord 2y ago
  Yes, indeed, the more they start sucking at things, the more money we should give them. This will incentivize not sucking.
  - Apocryphon 2y ago
    Hello, the spirit of No Child Left Behind

deleted 2y ago

[deleted]

mturmon 2y ago

Red hat at the end positions this as a job application?

lithos 2y ago
More like declaring the real plan of setting up one org that needs to be privatized. There have been real attempts at privatization of weather services in the US, despite the fact that our current system costs pennies a day per person (whether it's used for weather apps, or safe flying/sailing).
- momoschili 2y ago
  unlikely that there is good precedent for a private weather organization. The gold standard for western weather prediction is the European Centre for Medium-Range Weather Forecasts and that is a collaboration between multiple government agencies.
  - mrguyorama 2y ago
    The "lets privatize essential parts of the government" folks do not care about precedent, or what is a good idea. They are driven by ideology and self dealing.
    - momoschili 2y ago
      even Cliff himself (somewhat right leaning in today's American political environment) did not go so far in his blog. Cliff often speaks very highly of the European weather forecasts, and in his blog post it seems pretty clear that he is using it as the model for most of the suggestions.
      - throwway120385 2y ago
        The problem is that his well-meaning statements are going to be used as ammo to privatize NOAA, much like someone's well-meaning statements were used to create turmoil in Springfield, Missouri.

mnky9800n 2y ago

I just applied for a job at noaa after being asked by a director to apply for the job which was directly related to updating the core weather modelling libraries, some of which are 50+ years old Fortran, and also develop projects to integrate modern ai/ml into the modelling pipeline. I have 10+ years of experience working both as a researcher and a developer, a PhD in physics, and have won multiple grants either applying machine learning to geophysics problems or develop Bayesian spatiotemporal modeling libraries. At least on paper, I should get an interview for this job. I was rejected because: > We are not able to determine your qualifications as your resume does not show complete information for each job entry, such as beginning and ending dates of employment, duties performed, and/or total hours worked per week First, my cv did include everything but the total hours worked per week. Second there was no instruction that I could find for writing my CV such that I needed to list the number of hours worked per week. Third there was no way to contest this. Even the person who invited me to apply who would be the hiring manager had no control over this. I can tell you, this assuredly leads to the unnecessary decline of any system when there’s an impenetrable hr system sitting between applicants and jobs.

deleted 2y ago
[deleted]
brudgers 2y ago
Because NOAA is part of the Federal government, expect job applications to be inspected technically detailed relative to compliance with a mountain of Federal statutes and regulations. Because people who don't get hired can literally get their Congressional and Senate representatives involved. This makes getting hired into the Civil Service hard. Internal candidates are usually at an advantage. Veterans always are.
- mnky9800n 2y ago
  I’m currently a civil servant of Norway. They seem to have it figured out. Haha. But yeah I get you.
- warner25 2y ago
  > Internal candidates are usually at an advantage. Veterans always are. The system is as you described. Internal candidates already know it and veterans, when transitioning out of uniformed service, get a full-blown class on it: how to prepare their "Federal resume" (which is unlike any resume that a reasonable person might otherwise prepare) and navigate the hiring process. So it's that on top of already having relevant experience and credentials or preference points. I've been on the hiring side for Federal GS positions, and I'll tell you that it can be just as frustrating on that side too.

sampo 2y ago

> I have written two peer-reviewed papers in a major meteorological journal (here) The blogger is Cliff Mass, a professor at University of Washington. He has written two papers on this specific topic (decline of US weather predictions). In general, as a scientist, he has published 100+ papers. He only linked to his second paper [1]. I also found his first paper [2]. [1] "The Uncoordinated Giant II" (2023) https://journals.ametsoc.org/downloadpdf/view/journals/bams/104/4/BAMS-D-22-0037.1.pdf https://journals.ametsoc.org/downloadpdf/view/journals/bams/... [2] "The Uncoordinated Giant" (2006) https://www.e-education.psu.edu/files/meteo410/image/Lesson3/cliffmassBAMS0205.pdf https://www.e-education.psu.edu/files/meteo410/image/Lesson3...

deleted 2y ago
[deleted]

blueelephanttea 2y ago

I don't think the title of the article matches the content. The GFS has not declined relative to the ECMWF. It just started from a lower skill level and its improvements have not significantly closed the gap with ECMWF (as it has also improved). > Specifically, NOAA's global model, the UFS, is now in third or fourth place behind the European Center, the UK Meteorology Office, and often the Canadians. Would love to see evidence of that. It is well established that ECMWF is top in the game. I don't think it is reasonable to just state that Canada and UK are better without evidence. With that said, I agree the US should improve.

marcyb5st 2y ago
When it comes to medium range predictions. If you look at nowcasting, I believe the top dog is metnet 3 (https://arxiv.org/abs/2306.06079 https://arxiv.org/abs/2306.06079). On a tangential note, I worked on weather forecasting a bit before, and talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models. Not sure if they can't attract talent and so are somewhat stuck or if their higher ups are "old school" and can't really see the potential that this approach has to offer. It is just sad that the private sector is outclassing institutions and that these better models won't hit the public domain.
- plantain 2y ago
  > talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models This is trivially verifiable as not true. ECMWF is all over AI: https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-forecasting-system https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-... https://github.com/ecmwf-lab/ai-models https://github.com/ecmwf-lab/ai-models And they are hiring with competitive salaries (for Europe).
  - AStonesThrow 2y ago
    If only we'd put forecasting on the blonkchain so we could defend against 51% tornadoes

MrMcCall 2y ago

[flagged]

treyd 2y ago
This is also the same process that the Conservative parties in the UK ans Canada have been carrying out with their respective healthcare systems, much to the suffering of their citizens.
- MrMcCall 2y ago
  Indeed. One cannot get absurdly wealthy without making money from millions and millions of others. It's the nature of such wealth, and it is a sign of a society's moral decay. I'm not against making lots of money by providing a good product at a fair price, but that's not what right-wingers are doing. They are robber-barons robbing from the populace for their own selfish greed. And we are all left poorer, both literally and figuratively. Once we shift our personal and governmental focus to raising the standard of living for our least cared-for -- instead of allowing our systems to add more riches to the wealthy -- we can change this world into a fun, beautiful, and healthy place for us ALL. But that will not happen without selfless compassion. Selfish taking harms us all; selfless giving helps us all. To love someone is to want them to be happy, and we are in desperate need of far more lovingkindness in this world.

etiennebausson 2y ago

>> U.S. numerical weather prediction (NWP), which uses computer simulation to predict future weather, should be the best in the world. The author makes an important point, on a subject that keep getting more important due to climate change, but appeal to blind nationalism doesn't work in his favor. It's not so much a bias a willful blindness at this point. I have to tolerate it coming from politicians, but it is a different story when the author present himself as a serious scientist. Yes, the U.S. could be the best at meteorology. they could be the best at anything, with their means. But not at everything. The U.S. are the best at military spending, and everything else is at best in second place.

rossdavidh 2y ago
Well historically the military has quite often been one of the primary customers of weather forecasting, so that doesn't necessarily mean we couldn't have the best meteorology (assuming for the sake of argument that we don't).
- eximius 2y ago
  It doesn't seem likely that the military has a private model that is dramatically better. If we did, it would likely be exposed in a way similar to GPS.
  - itishappy 2y ago
    They'll release an intentionally kneecapped model until it's clear that workarounds are well known enough to allow release of the actual data? https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_Positioning_System#Selective_Availability https://en.wikipedia.org/wiki/Error_analysis_for_the_Global_...

jeffbee 2y ago

Interesting call for more horsepower behind numerical methods. One of the benefits of the emerging machine learning forecast models is dramatically lower need for computation (and therefore energy).

counters 2y ago
Sort of. The motivation here is that numerical weather prediction (NWP) models are used for a lot more than just weather forecasting - they're critical research infrastructure and tools. NWP allows researchers to simulate complex atmospheric flows and phenomena which might not be readily or directly observed - or to manipulate observed flows in ways to evaluate dynamical theory and the underlying physics. You can't really do any of this with the AI emulators, which output a tiny, tiny sliver of information relative to the vast array of data you can dump from an NWP system. Regarding energy usage, you're right up to a point. Today's ML weather models are trained to emulate very large reanalysis datasets (e.g., ECMWF's ERA5) which are produced using the physics-based models. No one has yet to demonstrate an end-to-end AI/ML weather forecast which sidesteps these reanalysis datasets (in fact, the trend so far has been to include _more_ physics-based model datasets for fine-tuning, including archives of historical NWP forecasts or GCM simulations). So there's a massive sunk cost in energy usage to create those training datasets, and then there's the ongoing energy cost of training AI emulators from the ground up. And of course, there's the future energy cost of running more physics-based models to better support the development of AI emulators. For pure inference/forecasting? Sure. Much faster to run AI models and much lower energy usage. But that's quite literally the tip of the iceberg when it comes to forecast model development.
- jeffbee 2y ago
  Sure, the AI stuff is not a complete solution because it relies on the results of numerical methods. I was only suggesting that in the question of making up the marginal difference between the best and fourth-best weather predictions, there might be some application of these other novel methods. If the question is can the Americans build a gigantic supercomputer and write fortran programs, the answer to that seems obvious. Yes we can do that when the relevant bureaucracy gets pointed in the right direction.

throw0101d 2y ago

Somewhat related, but I found Andrew Blum's book The Weather Machine: A Journey Inside the Forecast an interesting read: > In The Weather Machine, Andrew Blum takes readers on a fascinating journey through an everyday miracle. In a quest to understand how the forecast works, he visits old weather stations and watches new satellites blast off. He follows the dogged efforts of scientists to create a supercomputer model of the atmosphere and traces the surprising history of the algorithms that power their work. He discovers that we have quietly entered a golden age of meteorology—our tools allow us to predict weather more accurately than ever, and yet we haven’t learned to trust them, nor can we guarantee the fragile international alliances that allow our modern weather machine to exist. * https://www.goodreads.com/book/show/42079139-the-weather-machine https://www.goodreads.com/book/show/42079139-the-weather-mac... * https://www.andrewblum.net/the-weather-machine-2 https://www.andrewblum.net/the-weather-machine-2 Also his book Tubes, the Internet's physical infrastructure. He also appears to be working on a book on the electrical grid: * https://www.andrewblum.net/electric-earth https://www.andrewblum.net/electric-earth

warner25 2y ago
Thanks for the recommendation. I read Tubes back in 2012 when it was first published, and I enjoyed it. I knew a lot less about networking back then, so it would probably be a good book to re-read now with a different perspective, and to think about how things have might have changed since then.
- ForOldHack 2y ago
  Read Tubes last year. Magnificent read. A must read for internet users.
therein 2y ago
Aren't all weather forecasts getting their data from Raytheon in the US via Weather Central, LP?

M_bara 2y ago

Missed a chance to reword that epic as a failure to an epic failure

_yb2s 2y ago

As a sailor I’m really into getting a good weather prediction and most of the models are quite inaccurate. I learned on here about an ML based weather model startup that posted predictions on their website called Atmo.AI (edited) that was mind blowingly accurate and had incredible spacial resolution- it could accurately predict, e.g. the wind shadow behind islands, something none of the mainstream weather models even come close to doing. I used it everyday and it was almost always dead right at predicting wind speeds in almost any specific spot. I won a lot of sailboat races with it. I hope this tech gets taken up by NOAA, or other major weather predictors. Anyone on here know what happened to Atmo and if the tech is ever going to come back so people can use it? It's a travesty that this tech exists but isn't being used- a lot of lives would be saved if NOAA were using this already existing technology. Edit: Apparently as another poster pointed out I had just forgotten the correct name of it, and it is still around. I edited this to reflect that.

carabiner 2y ago
Do you mean atmo.ai? Atmos.ai seems to be something very different: "Atmos AI streamlines the Carbon Accounting, GHG Emissions, and Environmental, Social, and Governance (ESG) reporting processes for companies of all shapes and sizes."
- _yb2s 2y ago
  Ok, I am embarrassed, I simply forgot the name of it, and thought it had disappeared when I tried to find it again. Seems it is still alive and well. I swear this isn't viral marketing for them, I was legitimately confused. I edited my post above to use the correct name, but realize it doesn't make too much sense anymore.
code_biologist 2y ago
How do I use Atmo for locations other than the bay area demo linked on their website? I tried URL tweaking without much luck. I have a friend who does a lot of sailing races who would love this!
- 2y ago

jwr 2y ago

As a European, I can tell you not to worry too much. The EU forecasts are mostly garbage. For some reason (possibly climate change) weather forecasting became unreliable over the last 10-15 years. I regularly check 5 different forecasting models (through a subscription in the Windy app and elsewhere) and they are unable to agree on whether water will fall on my head later today. We're not talking long-range forecasting here, just tell me if it's going to rain in several hours or not. There is no forecasting model that gets this right.

magicalhippo 2y ago
Norwegian official weather forecasting service usually say they're quite accurate, and I was thinking how can that be? But in an interview last year they spilled the beans, they were talking about temperature and only missing by 1-2 degrees C most of the time. However, I don't care if it's 22C or 20C, like you I care about am I gonna need a jacket or not. I get that it's a difficult problem. And the inclusion of local weather radar to track actual rain has been very nice. But it only forecasts 90 minutes ahead, and rain can develop closer than that some times.

AIFounder 2y ago

[dead]

jccalhoun 2y ago

The article leaves out that members of one political party have proposed eliminating the NOAA entirely https://www.washingtonpost.com/weather/2024/07/22/project-2025-weather-service-trump/ https://www.washingtonpost.com/weather/2024/07/22/project-20...

yogurtboy 2y ago
Lol I shouldn't have clicked on that link, that shit really makes my blood boil. I don't think most people realize how much free value they get out of NOAA weather predictions.
- nickff 2y ago
  While I’d agree that accurate weather forecasts provide tremendous value, it’s not “free value”, as tax dollars are paying for it. That money could go to any number of ‘under-funded’ other causes, or be returned to the tax-payers.
  - aegypti 2y ago
    A relief to know NOAA isn’t doing this pro bono, I was beginning to wonder whether they were simply donating forecasts.
Spooky23 2y ago
This stuff has been a big bugaboo for GOP idiots for a long time. Rick Santorum (a former conservative luminary) was pushing for a law that would prohibit making weather data available to the public, but would make it available to companies like Accuweather for free. The brother of the Accuweather founder was Trump’s appointee to run the weather service. It’s on of those things that illustrates that no public resource is safe from graft.
tzs 2y ago
They also want to get rid of most civil service employees and change their positions to political appointees. Not just the top management positions, but also the people that actually do the day to day work. E.g., the scientists at the FDA who check the science on safety and effectiveness when processing approvals for new drugs would be political appointees who were vetted for loyalty to the President.

CapeTheory 2y ago

Don't worry, UK Met Office are currently shitting the bed on a migration from on-prem HPC to Azure (3+ years late at this point I believe?) so the competition isn't exactly hot.

joe_the_user 2y ago

Specifically, NOAA's global model, the UFS, is now in third or fourth place behind the European Center, the UK Meteorology Office, and often the Canadians. So there's been no decline. In fact US weather prediction is getting better (by the article own chart). It's just The author is upset the US isn't moving ahead as quickly as other weather services. Maybe they should be upset by this but the way it's framed seems a bit dishonest.

looping__lui 2y ago

One of the fee startups I am genuinely impressed: skysight. https://magazine.weglide.org/skysight-interview-matthew-scutter-gliding/ https://magazine.weglide.org/skysight-interview-matthew-scut... Freakingly accurate weather forecasts for the niche sport of gliding. Builds on top of ECMWF and incorporates refined soil moisture. It’s really good in predicting mountain waves (and used by Perlan: https://perlanproject.org/perlans-sponsors-enable-us-to-fly-high/ https://perlanproject.org/perlans-sponsors-enable-us-to-fly-... ). Also convergences and thermal activity which is very dynamic… Incredibly impressive.

timthorn 2y ago

> Our nation invented the technology... Lewis Fry Richardson might have something to say about that.

GarnetFloride 2y ago

I am an amateur radio operator and looking into the history of it, you quickly run into the telegraph, and telegraph operators would talk about things like the weather. It didn't take long before they realized that the weather in a place 100 miles away could show up the next day. Then they would warn everyone around them if bad weather was coming.

lisper 2y ago

I have lived in California for 36 years, 22 in SoCal and 14 in NorCal and I've noticed a trend that I find deeply disturbing: weather forecasts seem to be much less reliable now than they used to be. Before moving to California I lived on the east coast and the weather forecasts were generally unreliable, so I was really struck by how predictable the weather in California was, and how accurate the forecasts were. Rain began and ended when it was predicted to within a couple of hours. The temperature was within a few degrees of what it was predicted to be. And the window of predictability went out 5+ days. Nowadays that has all changed. Forecast rain fails to materialize. Heat waves come out of nowhere and last much longer and are much more intense than they are forecast to be. Temperatures at my house regularly go 10 degrees above what is predicted, and regularly hit temperatures that would have been extremely anomalous when we first moved here. This is all anecdotal. I haven't been keeping records, and I haven't done the math. It's possible that this is all confirmation bias or bad memory at work. But damn, it sure seems like a real phenomenon, and the most likely explanation is that the models were trained on data that don't reflect current conditions because of climate change. The most striking thing about this is the speed with which things are changing. 36 years is nothing on what should be geologic time scales. I shudder to think what the future holds.

rapjr9 2y ago
I've seen something similar happen and think I figured out the reason. Weather forecasts in my area were very accurate for decades, then they got much worse. They were off in timing for events, and incorrect about whether rain was going to fall or not. Then I realized that the change happened around the time the local nuclear power plant was shut down. The plant had been funding a local weather station and weather radio station as part of government requirements to provide early warning of problems to citizens. When the plant shut down the weather station went with it. So NOAA was no longer getting accurate data from this area and the forecasts were off because of that. I've been thinking of setting up my own station and linking it into the public data to see if that makes forecasts get more accurate.

snakeyjake 2y ago

>In most nations, numerical weather prediction is the responsibility of one group. In the U.S., governmental global weather prediction is spread over FIVE efforts: >NOAA, the National Oceanographic and Atmospheric Administration (the leading U.S. effort) >The US Navy >The US Air Force >NASA >U.S. Department of Energy This doesn't make sense. Other nations don't have a NASA, and if they do have a space agency that space agency is almost certainly in the weather business. JAXA: https://earth.jaxa.jp/en/data/2547/index.html https://earth.jaxa.jp/en/data/2547/index.html They also don't have expeditionary militaries like the USAF and USN, which need weather for the middle of the ocean or remote nation, far from NOAA's ability to deploy weather balloons. And as far as I can tell, the only exposure to weather that the DOE has is for climate change reasons, and their interest in that should be obvious. There's no waste. NOAA does the US, DOD does DOD, NASA helps out, and the DOE is looking at how climate change will impact the US's energy security. So reason number 1 is horseshit, and I stopped reading.

8bitsrule 2y ago

Mass/Maas uses European results in his blog often, often commenting that the European predictions for the PNW are more accurate. So, fine, that's great. Why does the US have to keep up? Will it stop the decline of the Empire? Is it a stab in the heart of our world posture? Are we doomed to having to be the best at everything for all of eternity? I don't get it. I'm more upset about all the interest being paid on the US national debt that -could- have been spent on something great instead.

paulkon 2y ago

“In the U.S., governmental global weather prediction is spread over FIVE efforts: NOAA The US Navy The US Air Force NASA U.S. Department of Energy Thus, scientific and technical talent and computer resources are diffused over five groups, greatly undermining progress.“ I suppose if Elon has his way, the overlapping functions from these are on the chopping block.

dramm 2y ago

Run your own code :-) I used to fly sailplanes and around Northern California at least there is an interesting intersection of computer nerds, aerodynamics nerds and weather nerds. One of my favorite things being Jack Glendening's weather code/models for sailplane pilots, culminating in the RASP forecasts that more technical sailplane (and hang glider etc) pilots ran for their own hyper-local forecasts. The localized terrain triggered thermal hotspots and convergence prediction of these RASP models were super impressive. http://www.drjack.info/rasp http://www.drjack.info/rasp. Today specialized commercial forecast services are used by many sailplane pilots, and RASP and related forecasts predated or inspired those, but a well run RASP forecast on a fine grid is still something to behold.