6 ms·

Not sure I entirely agree? #1 > Or disable a hospital. The entire Ascension Healthcare system of hospitals (142 hospitals, 2600 total facilities) in on divert

Not sure I entirely agree? #1 > Or disable a hospital. The entire Ascension Healthcare system of hospitals (142 hospitals, 2600 total facilities) in on divert since 8 May because they had to switch back to paper records. Change Healthcare has lost $872M since it was attacked in February. Maybe it's more like the pandemic: seems like nothing, unless it affects you. https://en.wikipedia.org/wiki/Ascension_(healthcare_system) https://en.wikipedia.org/wiki/Ascension_(healthcare_system) https://www.wired.com/story/change-healthcare-admits-it-paid-ransomware-hackers/ https://www.wired.com/story/change-healthcare-admits-it-paid... #2 > Does your stuff need computers working 5,000 kilometers away? [implying that's bad] What if you live on the Gulf Coast, exposed to hurricanes? You want compute resources warm and ready far away from that region. After Katrina, the Tulane medical school was able to re-form quickly because the noteservice was running a bulletin board forum on a VM in Romania. Everything else was underwater. #3 > This is the sound-powered phone Have you used a sound-powered phone? I managed damage control in a ship. Sound powered phones barely works. And the coordination system to actually fight that fire requires radios and making overhead announcements that definitely depend on electrical power. #4 > They tried to sort of renew this emergency telephone network When the entire San Diego region lost power during rush hour for 4 hours in 2011, the cell phone system still worked. I was able to email documents to Tokyo from a car despite no traffic lights. #5 > Because if the cable to the US is down Sure, but there are a lot of disasters where the cables are fine. Graceful degradation is all about having widely distributed options. Lots of people have What. Signal is even better for people with more serious responsibilities, IMHO. And, friends, if you think IP networks are vulnerable, get yourself a starlink terminal and a HAM radio license. https://en.wikipedia.org/wiki/SpaceX_Starshield https://en.wikipedia.org/wiki/SpaceX_Starshield

indymike 2y ago
> Change Healthcare has lost $872M since it was attacked in February. The question is, what is the cost to secure? I've been in so many meetings where the cost of security is 10-15x the cost of a breach. It's horrifying.
- stouset 2y ago
  Part of this is that nobody has cared about security since the beginning, for basically anything in tech. It’s an industry-wide issue that permeates every level of the stack. And so yeah, individual companies trying to retrofit security onto a jenga tower of technology is going to have to spend a ridiculous amount of resources to have any kind of impact. I don’t know what the answer is, but I too believe things won’t change until the day someone figures out how to push a “kill all humans” OTA update to all the self-driving cars on some random Tuesday afternoon.
  - ryandrake 2y ago
    > I don’t know what the answer is, but I too believe things won’t change until the day someone figures out how to push a “kill all humans” OTA update to all the self-driving cars on some random Tuesday afternoon. Even in that case I’m pessimistic that any action will happen. People will go on TV and say grave things, hearings will be held. Fingers will be pointed. Task Forces will kick off. Reports will be written. Bureaucrats will have stern conversations with bureaucrats. Politicians will say: we must this and we shall that. IT companies will sell their “solutions”. But no actual action will happen. It will be all talk and commerce but no actual hands unplugging and plugging in cables. We have completely lost the societal will to actually do anything besides generate words and reports.
    - eastbound 2y ago
      You are describing the current world, where politicians dissolve issues. There’s a saying in Europe that no minister of defense was ever nominated. Real ministers of war, when there is war, appoint themselves into position. When there is a real problem, people act upon it (assuming society is functional - otherwise the country simply dies). That’s why there is no better training for war than war itself. Ukraine has already unrooted all of the peace & love & no armament folklore in France, and even turned a lot of ecologists into pro-nuclear voters. So yes, I wouldn’t be surprised if guarantees of offline mode (with regular drills) were passed into law for electric cars and everything cloudy, including IntelliJ.
  - 2y ago
dralley 2y ago
TL;DR hybrid-cloud, multi-cloud, or at the very minimum multi-region is a really good idea.
freehorse 2y ago
> but there are a lot of disasters where the cables are fine We are talking about war-like situations, and where one state actor has incentive to cause maximum harm to another. Exposing your infrastructure like this is unlike damage that can come from natural disaster. For example, disrupting the communications exactly before the attack. Similar issues (though through lower tech hacking) happened in 7th of October during the Hamas attack in Israel, where the over-reliance on advanced, complicated technology became a liability. The stuff you describe make sense in normal, peaceful situations, where the cost of securing certain infrastructure can be higher than the cost of a power cut once. That has nothing to do with what the article really says, which is basically that infrastructure is currently not as secure from a potential hostile state attack. Also, in that case, a hostile state actor can combine attacks that together cause more damage than the sum of the attacks independently.
- yardstick 2y ago
  What was the lower tech stuff on Oct 7?
  - alephnerd 2y ago
    There was a massive DDoS in Israel on 10/7, fake alerts of a nuclear missile launch were sent, and newspapers like the Jerusalem Post were taken down and temporarily defaced[0] [0] - https://www.axios.com/2023/10/10/hackers-ddos-israel-hamas-conflict https://www.axios.com/2023/10/10/hackers-ddos-israel-hamas-c...
hughesjj 2y ago
The back of my head is screaming "defense in depth! Redundant systems!" The whole idea of the internet (and even some of our infra, like suburbs or highways/rail) is that there's no one single point of failure. Like designed-to-survive-nuclear-war redundant. Definitely incorporate the most advanced tech you can for when things are going smoothly to get that efficency gain, but there's a reason all branches of the military (that I'm aware of) still train and test their aptitude using paper maps and trig instead of relying 100% on GPS and electronic devices.
- Dalewyn 2y ago
  >The whole idea of the internet (and even some of our infra, like suburbs or highways/rail) is that there's no one single point of failure. Like designed-to-survive-nuclear-war redundant. The reality of course is that the internet has turned into a fragile, centralized system of complication that rests on single failure points like Cloudflare, AWS, and Chrome. The internet as envisioned by DARPA would have survived to be used by cockroaches, the internet today would not survive.
  - BLKNSLVR 2y ago
    "The Internet" is still there at a lower level than the examples you've given. It's just how it's been used by the majority for the last couple of decades that's fragile.
  - wbl 2y ago
    Chrome works even if Google is down How is it an SPOF? And links mozilla.org will get you an alternative.
  - theeandthy 2y ago
    Yeah it’s pretty bad nowadays. Thinking about this though it’s really the big tech companies manufacturing “the latest thing” to be tossed in the bin after a year. Dollars over longevity. Then they become “no longer maintained.” Could we STILL use a 3g network? Or is there a simpler, slow network that should be good enough barring our pointless desire for cat videos? And some folks wonder why companies still use floppy disks on air-gapped infrastructure. Because it fucking works don’t litter it with complexity to modernize. Now… the situation with skills to manage infrastructure? Now that the whole AI thing is happening? The internet is going to be fucked people. It’s time to go analog.
baxtr 2y ago
I find it additionally odd that the author calls this era pre war. Ukraine is certainly at war right now with a very potent cyber state. Their infrastructure seems to hold up ok. It’s not perfect but definitely not doomsday like described in this article.
- fullspectrumdev 2y ago
  Tbf their infra holds up because their infrastructure workers put their lives on the line every single day repairing it under horrible conditions of shelling, etc. On my most recent trip there - I was amazed at how despite being routinely hit by missiles, their train systems “on time” status is better than British or even German trains. This is only possible because their railway workers have balls of steel and go out to repair damage fast, and sometimes get hit in follow up strikes. Same with energy workers - they go out and repair stuff during air alarms, in the immediate aftermath of strikes they perform damage control and mitigations.
- int_19h 2y ago
  It's still a pre-war era for the Netherlands.
alephnerd 2y ago
On the hospital system part, there are actual timelines and goals to harden their systems after seeing what happened to the HSE in 2021. The issue is some parts of the chain have been slow on the uptick. That said, paper based redundancies do exist as a massive ransomware attack is similar in impact to a multiweek power outage.
GuB-42 2y ago
[flagged]
toast0 2y ago
> When the entire San Diego region lost power during rush hour for 4 hours in 2011, the cell phone system still worked. I was able to email documents to Tokyo from a car despite no traffic lights. Around me, cell towers have 3-5 hours of battery when utility power is out. If your outage had gone on much longer, you would likely have seen cell towers start dropping out. Of course, my area also has some other nasty SPoFs. A couple years ago, a telco cable was severed and DSL for everyone was out and at least some of the cell towers were live, but no service. A few weeks ago, the cableco had its wires severed, and cable tv and internet was offline, and so were some cell towers. IIRC, for the telco one t-mobile worked and verizon didn't, and for the cableco t-mobile didn't work and verizon did. Not sure about at&t.
pyrale 2y ago
> What if you live on the Gulf Coast, exposed to hurricanes? [...] Sure, but there are a lot of disasters where the cables are fine. You have to understand that this article was written by an European technologist worrying about a war situation. Sure, you can make a counter-point, but your counter-example is very different in many aspects: nature of the threat, jurisdictions involved, orgs involved, etc.

indymike 2y ago

> Change Healthcare has lost $872M since it was attacked in February. The question is, what is the cost to secure? I've been in so many meetings where the cost of security is 10-15x the cost of a breach. It's horrifying.

stouset 2y ago
Part of this is that nobody has cared about security since the beginning, for basically anything in tech. It’s an industry-wide issue that permeates every level of the stack. And so yeah, individual companies trying to retrofit security onto a jenga tower of technology is going to have to spend a ridiculous amount of resources to have any kind of impact. I don’t know what the answer is, but I too believe things won’t change until the day someone figures out how to push a “kill all humans” OTA update to all the self-driving cars on some random Tuesday afternoon.
- ryandrake 2y ago
  > I don’t know what the answer is, but I too believe things won’t change until the day someone figures out how to push a “kill all humans” OTA update to all the self-driving cars on some random Tuesday afternoon. Even in that case I’m pessimistic that any action will happen. People will go on TV and say grave things, hearings will be held. Fingers will be pointed. Task Forces will kick off. Reports will be written. Bureaucrats will have stern conversations with bureaucrats. Politicians will say: we must this and we shall that. IT companies will sell their “solutions”. But no actual action will happen. It will be all talk and commerce but no actual hands unplugging and plugging in cables. We have completely lost the societal will to actually do anything besides generate words and reports.
  - eastbound 2y ago
    You are describing the current world, where politicians dissolve issues. There’s a saying in Europe that no minister of defense was ever nominated. Real ministers of war, when there is war, appoint themselves into position. When there is a real problem, people act upon it (assuming society is functional - otherwise the country simply dies). That’s why there is no better training for war than war itself. Ukraine has already unrooted all of the peace & love & no armament folklore in France, and even turned a lot of ecologists into pro-nuclear voters. So yes, I wouldn’t be surprised if guarantees of offline mode (with regular drills) were passed into law for electric cars and everything cloudy, including IntelliJ.
- 2y ago

dralley 2y ago

TL;DR hybrid-cloud, multi-cloud, or at the very minimum multi-region is a really good idea.

freehorse 2y ago

> but there are a lot of disasters where the cables are fine We are talking about war-like situations, and where one state actor has incentive to cause maximum harm to another. Exposing your infrastructure like this is unlike damage that can come from natural disaster. For example, disrupting the communications exactly before the attack. Similar issues (though through lower tech hacking) happened in 7th of October during the Hamas attack in Israel, where the over-reliance on advanced, complicated technology became a liability. The stuff you describe make sense in normal, peaceful situations, where the cost of securing certain infrastructure can be higher than the cost of a power cut once. That has nothing to do with what the article really says, which is basically that infrastructure is currently not as secure from a potential hostile state attack. Also, in that case, a hostile state actor can combine attacks that together cause more damage than the sum of the attacks independently.

yardstick 2y ago
What was the lower tech stuff on Oct 7?
- alephnerd 2y ago
  There was a massive DDoS in Israel on 10/7, fake alerts of a nuclear missile launch were sent, and newspapers like the Jerusalem Post were taken down and temporarily defaced[0] [0] - https://www.axios.com/2023/10/10/hackers-ddos-israel-hamas-conflict https://www.axios.com/2023/10/10/hackers-ddos-israel-hamas-c...

hughesjj 2y ago

The back of my head is screaming "defense in depth! Redundant systems!" The whole idea of the internet (and even some of our infra, like suburbs or highways/rail) is that there's no one single point of failure. Like designed-to-survive-nuclear-war redundant. Definitely incorporate the most advanced tech you can for when things are going smoothly to get that efficency gain, but there's a reason all branches of the military (that I'm aware of) still train and test their aptitude using paper maps and trig instead of relying 100% on GPS and electronic devices.

Dalewyn 2y ago
>The whole idea of the internet (and even some of our infra, like suburbs or highways/rail) is that there's no one single point of failure. Like designed-to-survive-nuclear-war redundant. The reality of course is that the internet has turned into a fragile, centralized system of complication that rests on single failure points like Cloudflare, AWS, and Chrome. The internet as envisioned by DARPA would have survived to be used by cockroaches, the internet today would not survive.
- BLKNSLVR 2y ago
  "The Internet" is still there at a lower level than the examples you've given. It's just how it's been used by the majority for the last couple of decades that's fragile.
- wbl 2y ago
  Chrome works even if Google is down How is it an SPOF? And links mozilla.org will get you an alternative.
- theeandthy 2y ago
  Yeah it’s pretty bad nowadays. Thinking about this though it’s really the big tech companies manufacturing “the latest thing” to be tossed in the bin after a year. Dollars over longevity. Then they become “no longer maintained.” Could we STILL use a 3g network? Or is there a simpler, slow network that should be good enough barring our pointless desire for cat videos? And some folks wonder why companies still use floppy disks on air-gapped infrastructure. Because it fucking works don’t litter it with complexity to modernize. Now… the situation with skills to manage infrastructure? Now that the whole AI thing is happening? The internet is going to be fucked people. It’s time to go analog.

baxtr 2y ago

I find it additionally odd that the author calls this era pre war. Ukraine is certainly at war right now with a very potent cyber state. Their infrastructure seems to hold up ok. It’s not perfect but definitely not doomsday like described in this article.

fullspectrumdev 2y ago
Tbf their infra holds up because their infrastructure workers put their lives on the line every single day repairing it under horrible conditions of shelling, etc. On my most recent trip there - I was amazed at how despite being routinely hit by missiles, their train systems “on time” status is better than British or even German trains. This is only possible because their railway workers have balls of steel and go out to repair damage fast, and sometimes get hit in follow up strikes. Same with energy workers - they go out and repair stuff during air alarms, in the immediate aftermath of strikes they perform damage control and mitigations.
int_19h 2y ago
It's still a pre-war era for the Netherlands.

alephnerd 2y ago

On the hospital system part, there are actual timelines and goals to harden their systems after seeing what happened to the HSE in 2021. The issue is some parts of the chain have been slow on the uptick. That said, paper based redundancies do exist as a massive ransomware attack is similar in impact to a multiweek power outage.

GuB-42 2y ago

[flagged]

toast0 2y ago

> When the entire San Diego region lost power during rush hour for 4 hours in 2011, the cell phone system still worked. I was able to email documents to Tokyo from a car despite no traffic lights. Around me, cell towers have 3-5 hours of battery when utility power is out. If your outage had gone on much longer, you would likely have seen cell towers start dropping out. Of course, my area also has some other nasty SPoFs. A couple years ago, a telco cable was severed and DSL for everyone was out and at least some of the cell towers were live, but no service. A few weeks ago, the cableco had its wires severed, and cable tv and internet was offline, and so were some cell towers. IIRC, for the telco one t-mobile worked and verizon didn't, and for the cableco t-mobile didn't work and verizon did. Not sure about at&t.

pyrale 2y ago

> What if you live on the Gulf Coast, exposed to hurricanes? [...] Sure, but there are a lot of disasters where the cables are fine. You have to understand that this article was written by an European technologist worrying about a war situation. Sure, you can make a counter-point, but your counter-example is very different in many aspects: nature of the threat, jurisdictions involved, orgs involved, etc.