4 ms·

What this also tells me is there are a lot of computers connected to the internet that probably shouldn’t be.

0cf8612b2e1e 2y ago
I suspect it is incredibly challenging to keep a non trivial number of computers available, but somewhat airgapped. Too many ways to unintentionally bridge the networks without extreme diligence. Which is slightly incompatible with something like airlines which employ enormous numbers of people.
- azinman2 2y ago
  These are not small time operations (most of them). They are multi billion dollar companies with complex technical needs. This is doable and the bread and butter of good networking engineers. Having not done this just cost them billions. Now imagine the US at war or another nation state that just wants to cause havoc.
  - unethical_ban 2y ago
    Crowdstrike is a trusted application on every computer in these people's farms. It is not uncommon to have specific rules for these packages to be downloaded directly from the Internet. You're suggesting either that Crowdstrike itself will get used as a vessel for an attack, or that banks and airlines have firewall rules open for enemies.
  - nradov 2y ago
    That's ridiculous. The cost of building air gapped operations systems, and including the resulting loss of productivity and efficiency, would be worse than just accepting an occasional outage. People who lack technical competence and operational experience always tend to over react to software failures. The major airlines may not be "small time operations" but none of them are even in the top 100 US corporations by market cap. They simply don't have the level of IT resources or competence that we see at major tech companies.
    - oceanplexian 2y ago
      It doesn’t need to be strictly air gapped, but there’s nothing technologically demanding about a computer used to check people in on a flight, it could be done over a 9600 baud modem and a thin client.
      - macintux 2y ago
        Nothing technologically demanding about global logistics software, being run by an industry with 0% profit margins (give or take, airlines don’t make much money), being asked to cripple itself 24x7 to avoid one weird outage every several years?
PenguinCoder 2y ago
Air-gapped networks have gotten more scarce in the day and age of the cloud computing. Expect for certain DoD and cleared spaces, I've even seen PLC networks internet connected....
hypeatei 2y ago
Exactly, most of these systems probably wouldn't require EDR software if networking was done correctly in the first place.
rainsford 2y ago
It's an interesting thought experiment to consider everything that would have to go into running operations for a business like one of the largest airlines in the world using a non-Internet connected network. Among other things, you probably lose the ability for your employees who aren't physically in the office (which is kind of a lot of them if you're running an airline) from interacting with your operations network. If you're an airline trying to schedule employees and share information with them while they're in hotel rooms, that's probably a deal breaker. That's really only a secondary problem though, because disconnecting a network from the Internet isn't a replacement for security software or software updates, so you wouldn't even avoid the root cause of the issue here. I'm not saying CrowdStrike is essential software for Internet connected computers either, but if your business thinks it is, you should probably be running it on your "airgapped" computers too. And you should definitely be installing updates, so you can still fall victim to a bad updates regardless of which software you run. At best you perhaps increase the likelihood of hearing about a problem with an update before you deploy it on disconnected computers, but you can get a similar effect by delayed deployment of updates even on Internet connected networks.
filleokus 2y ago
Hmm. I think this is a pretty shallow take. My experience from the airline industry is that the vast majority of systems classified as flight safety critical are not connected to the internet, or large networks at all. Which is good. But unless we want to drastically change how airlines operate, the rest need to be online. Today, you can purchase a ticket (or rebook an existing one) on your phone really close to the departure time. When that happens, a gazillion interconnected systems, across legal entity borders, need to cooperate to take you (and your luggage) to the destination. To put all of this in a large non-internet network seems pretty pointless. If we wanna go down that route, the only real "security improvement" I can think of is to dismantle the digital systems and go back to paper. Like Ryanair did during this incident. Handwritten boarding passess, verified against print-outs of passenger manifests.
- tgv 2y ago
  > the vast majority ... are not connected to the internet But those couldn't have gone down due to Crowdstrike.
shagie 2y ago
The related question is "How do you run your business out of downtown Fort Worth (American Airlines) and get your updates to 350 airports in 60 countries?" Saying "run your own network" isn't exactly practical. Even imagining the very small airlines (that partner with big ones for the last leg) that only service a handful of rural airports, this doesn't seem practical. The days of point to point updates over modems are overish with the amount of data that needs to be consistent transmitted and available. I can imagine a modem at each airport and a phone bank of about 700 modems that are each getting or sending updates. The long distance calls to distant countries for that data could get expensive. Woe to the power outage in Texas that takes down the phone bank for a day or two or three or four. Alternatively, there's a system that was developed to do this and it works pretty well most of the time. Combine this with having redundant systems there that are geographically separated. It isn't turnkey, but its probably better than other options that would involve home grown solutions.
deleted 2y ago
[deleted]
Ekaros 2y ago
So Web 2.0 was a mistake I take? Problem once again is humans. Humans need to interact with systems, either receive information from them or give it to them or use the systems to process it. And for efficiency in general it nowadays happens online. It could be offline, but that would be slow. Or it could be segregated networks but that would get really expensive. Imagine having own fiber line for your instant messaging and email? With different terminal... In the end most of these affected computers are on Internet for very good reasons. And this model really is working vast majority of time generating lot of efficiency.

I suspect it is incredibly challenging to keep a non trivial number of computers available, but somewhat airgapped. Too many ways to unintentionally bridge the networks without extreme diligence. Which is slightly incompatible with something like airlines which employ enormous numbers of people.

azinman2 2y ago
These are not small time operations (most of them). They are multi billion dollar companies with complex technical needs. This is doable and the bread and butter of good networking engineers. Having not done this just cost them billions. Now imagine the US at war or another nation state that just wants to cause havoc.
- unethical_ban 2y ago
  Crowdstrike is a trusted application on every computer in these people's farms. It is not uncommon to have specific rules for these packages to be downloaded directly from the Internet. You're suggesting either that Crowdstrike itself will get used as a vessel for an attack, or that banks and airlines have firewall rules open for enemies.
- nradov 2y ago
  That's ridiculous. The cost of building air gapped operations systems, and including the resulting loss of productivity and efficiency, would be worse than just accepting an occasional outage. People who lack technical competence and operational experience always tend to over react to software failures. The major airlines may not be "small time operations" but none of them are even in the top 100 US corporations by market cap. They simply don't have the level of IT resources or competence that we see at major tech companies.
  - oceanplexian 2y ago
    It doesn’t need to be strictly air gapped, but there’s nothing technologically demanding about a computer used to check people in on a flight, it could be done over a 9600 baud modem and a thin client.
    - macintux 2y ago
      Nothing technologically demanding about global logistics software, being run by an industry with 0% profit margins (give or take, airlines don’t make much money), being asked to cripple itself 24x7 to avoid one weird outage every several years?

PenguinCoder 2y ago

Air-gapped networks have gotten more scarce in the day and age of the cloud computing. Expect for certain DoD and cleared spaces, I've even seen PLC networks internet connected....

hypeatei 2y ago

Exactly, most of these systems probably wouldn't require EDR software if networking was done correctly in the first place.

rainsford 2y ago

It's an interesting thought experiment to consider everything that would have to go into running operations for a business like one of the largest airlines in the world using a non-Internet connected network. Among other things, you probably lose the ability for your employees who aren't physically in the office (which is kind of a lot of them if you're running an airline) from interacting with your operations network. If you're an airline trying to schedule employees and share information with them while they're in hotel rooms, that's probably a deal breaker. That's really only a secondary problem though, because disconnecting a network from the Internet isn't a replacement for security software or software updates, so you wouldn't even avoid the root cause of the issue here. I'm not saying CrowdStrike is essential software for Internet connected computers either, but if your business thinks it is, you should probably be running it on your "airgapped" computers too. And you should definitely be installing updates, so you can still fall victim to a bad updates regardless of which software you run. At best you perhaps increase the likelihood of hearing about a problem with an update before you deploy it on disconnected computers, but you can get a similar effect by delayed deployment of updates even on Internet connected networks.

filleokus 2y ago

Hmm. I think this is a pretty shallow take. My experience from the airline industry is that the vast majority of systems classified as flight safety critical are not connected to the internet, or large networks at all. Which is good. But unless we want to drastically change how airlines operate, the rest need to be online. Today, you can purchase a ticket (or rebook an existing one) on your phone really close to the departure time. When that happens, a gazillion interconnected systems, across legal entity borders, need to cooperate to take you (and your luggage) to the destination. To put all of this in a large non-internet network seems pretty pointless. If we wanna go down that route, the only real "security improvement" I can think of is to dismantle the digital systems and go back to paper. Like Ryanair did during this incident. Handwritten boarding passess, verified against print-outs of passenger manifests.

tgv 2y ago
> the vast majority ... are not connected to the internet But those couldn't have gone down due to Crowdstrike.

shagie 2y ago

The related question is "How do you run your business out of downtown Fort Worth (American Airlines) and get your updates to 350 airports in 60 countries?" Saying "run your own network" isn't exactly practical. Even imagining the very small airlines (that partner with big ones for the last leg) that only service a handful of rural airports, this doesn't seem practical. The days of point to point updates over modems are overish with the amount of data that needs to be consistent transmitted and available. I can imagine a modem at each airport and a phone bank of about 700 modems that are each getting or sending updates. The long distance calls to distant countries for that data could get expensive. Woe to the power outage in Texas that takes down the phone bank for a day or two or three or four. Alternatively, there's a system that was developed to do this and it works pretty well most of the time. Combine this with having redundant systems there that are geographically separated. It isn't turnkey, but its probably better than other options that would involve home grown solutions.

deleted 2y ago

[deleted]

Ekaros 2y ago

So Web 2.0 was a mistake I take? Problem once again is humans. Humans need to interact with systems, either receive information from them or give it to them or use the systems to process it. And for efficiency in general it nowadays happens online. It could be offline, but that would be slow. Or it could be segregated networks but that would get really expensive. Imagine having own fiber line for your instant messaging and email? With different terminal... In the end most of these affected computers are on Internet for very good reasons. And this model really is working vast majority of time generating lot of efficiency.