3 ms·

Your point may well be an important aspect of this. But to the extent that's true, we shouldn't be throwing money at a system (in the larger sense, not just NWS

Your point may well be an important aspect of this. But to the extent that's true, we shouldn't be throwing money at a system (in the larger sense, not just NWS) that's broken. Unless of funding, let's find the badly-designed regulation and fix that. But it's more than just NWS Chat: In December, because of an escalating bandwidth shortage, the Weather Service proposed limiting users to 60 connections per minute on a large number of its websites. Constituents complained about the quota and, earlier this month, the Weather Service announced it would instead impose a data limit of 120 requests per minute and only on servers hosting model data, beginning April 20. This is, or should be, bread and butter stuff. The rest of the world has figured out how to do this cheaply and with high availability. And their overseers even recognize this: Neil Jacobs, former acting head of the National Oceanic Atmospheric Administration, which oversees the Weather Service, said many of the agency’s Internet infrastructure problems are tied to the fact they run on internal hardware rather than through cloud service providers such as Amazon Web Services, Microsoft and Google Cloud. “I’ve demanded in writing that NWS transition these applications … to our Cloud partners. It’s part of an internal strategy I’ve laid out,” Jacobs, a Trump administration appointee, told the Capital Weather Gang in an email before he left office. So why are they not making that architectural change?

yellowapple 6y ago
I ain't really convinced by Jacobs' take; if they're having these issues with on-prem hardware, then they're going to have these same issues with cloud services. It's a matter of resource allocation, effort, and the organization itself, not one of specific implementation details like where the servers live. A cloud transition sounds like a boondoggle and a distraction from more fundamental issues - and when (not if) those issues persist, now we'll be in the same boat but with even worse results per tax dollar.
Jtsummers 6y ago
> So why are they not making that architectural change? Because IT is secondary to their primary mission and they don't have a proper dedicated IT team of sufficient size to enable this or a smooth transition if they tried. This is a repeated problem in the federal government. IT is not their mission, and each entity acts like a small-to-medium sized business, even if they're part of a larger scale organization. The US DOD doesn't even have one email system for all services. Each service has its own, and at least until the early 2010s many (most? all?) military bases had their own email servers with military members (and civilians) having to get new email accounts at each new base they worked at (this isn't the case of having a distributed but mutually coordinated system, it was just that each base literally had their own MS Exchange servers running and each maintained them themselves). And that's pretty much the single largest entity in the executive. Now imagine if it was a struggle for them despite a clear mission benefit how much more challenging this is when the smaller agencies are asked to do the same without being provided sufficient personnel and finances to do it, and no leadership (no real leadership) stepping up to create or establish a cross-service solution.
- CobaltFire 6y ago
  I can’t speak to the other services but this wasn’t true for the Department of the Navy from about 2005 on. There were geographic domains (@navy.mil, @usmc.mil, @fe.navy.mil, etc.) that required you to get new accounts, but other than that you were within NMCI. Then NMCI got sold out via contracts that underbid and incrementally removed functionality. Now you can’t recover an account you previously had because they aren’t required to save the account data for more than 90 days (cost cutting) and they don’t recycle addresses due to spillage. Finding anyone without a number attached to their email address has become pretty rare due to that. Now you have two primary systems in the Navy (OneNet for overseas and NMCI for otherwise) and the USMC has basically broken their network off from the Navy. Further, each ship has to have its own email servers for fairly obvious reasons. Kind of rambling, but I’ve worked with that system for a long time now. It’s been interesting going from the legacy military systems, to what was an excellent system, then watching it decay into what it is today.

yellowapple 6y ago

I ain't really convinced by Jacobs' take; if they're having these issues with on-prem hardware, then they're going to have these same issues with cloud services. It's a matter of resource allocation, effort, and the organization itself, not one of specific implementation details like where the servers live. A cloud transition sounds like a boondoggle and a distraction from more fundamental issues - and when (not if) those issues persist, now we'll be in the same boat but with even worse results per tax dollar.

Jtsummers 6y ago

> So why are they not making that architectural change? Because IT is secondary to their primary mission and they don't have a proper dedicated IT team of sufficient size to enable this or a smooth transition if they tried. This is a repeated problem in the federal government. IT is not their mission, and each entity acts like a small-to-medium sized business, even if they're part of a larger scale organization. The US DOD doesn't even have one email system for all services. Each service has its own, and at least until the early 2010s many (most? all?) military bases had their own email servers with military members (and civilians) having to get new email accounts at each new base they worked at (this isn't the case of having a distributed but mutually coordinated system, it was just that each base literally had their own MS Exchange servers running and each maintained them themselves). And that's pretty much the single largest entity in the executive. Now imagine if it was a struggle for them despite a clear mission benefit how much more challenging this is when the smaller agencies are asked to do the same without being provided sufficient personnel and finances to do it, and no leadership (no real leadership) stepping up to create or establish a cross-service solution.

CobaltFire 6y ago
I can’t speak to the other services but this wasn’t true for the Department of the Navy from about 2005 on. There were geographic domains (@navy.mil, @usmc.mil, @fe.navy.mil, etc.) that required you to get new accounts, but other than that you were within NMCI. Then NMCI got sold out via contracts that underbid and incrementally removed functionality. Now you can’t recover an account you previously had because they aren’t required to save the account data for more than 90 days (cost cutting) and they don’t recycle addresses due to spillage. Finding anyone without a number attached to their email address has become pretty rare due to that. Now you have two primary systems in the Navy (OneNet for overseas and NMCI for otherwise) and the USMC has basically broken their network off from the Navy. Further, each ship has to have its own email servers for fairly obvious reasons. Kind of rambling, but I’ve worked with that system for a long time now. It’s been interesting going from the legacy military systems, to what was an excellent system, then watching it decay into what it is today.