4 ms·

I think that's a false analogy. I don't have a clear sense on the ethics here, but the basic point is that we should be resilient to crises and broad cyberattac

by blagie 3y ago

I think that's a false analogy. I don't have a clear sense on the ethics here, but the basic point is that we should be resilient to crises and broad cyberattacks. Cell towers _will_ sometimes go down, and rare events, such as nation-wide cyberattacks, _will_ eventually happen. Imagine if [adversary] spends time finding a zero-day in iOS, Android, Linux, Windows, and MacOS, and releases a worm which bricks every computer it gets on (e.g. overwrites every firmware with something maximally malicious). That's within the scope of capability of many world governments. What happens? Do people in hospitals die? Does out infrastructure collapse? Or do things keep ticking on, somehow? Tools like Chaos Monkey intentionally introduce more errors under normal operating circumstances in order to make systems more robust in extreme ones. This is a real-world analogy. For something like this on a train, it's a question of value systems. I can't bring myself to _have_ a value system which makes this okay for me to do something similar, but I can see rational value systems which would allow that (e.g. hedonistic utilitarianism). It's the trolly problem. All the examples you gave, in contrast, are just victim-blaming. There is no greater good. By my value system, though, what I would like to see are similar tests done planfully and intentionally. In the abstract, if a federal government took down the internet and cell phone networks for eight hours, or we had planned power blackouts, or similar, I'd be fully supportive. That would require organizations to build in the right types of resiliency. I understand that's fully impossible in the kinds of political systems we have. But in the right political system, I'd like to have a government which works to make us resilient to those kinds of extreme events.

ethanbond 3y ago
It's not a false analogy at all. You just happen to care about cyber risk and resilience, so you think it's okay to accept a higher cost in pursuit of it. Here you go: * You need steel security doors because we should be resilient to crises and broad security threats. You don't know when there will be a major gang war, a neighbor with a psychotic break, a government breakdown, or a ground invasion. It's just better to have steel security doors because there are all sorts of real risks that they mitigate. * You need to drive slowly because some day someone will pull out in front of you, even by accident! Accidents like that happen hundreds or thousands of times per day, so it's frankly insane not to be prepared for it. * You need to check all your restaurant's ingredients because someone will forget to throw out the milk one day. We need to be resilient to this because it's certainly going to happen on occasion, and the right way to achieve resilience is to shift that burden onto end users, or at least to continually poison end users until they demand action from food regulators that we have a 100% foolproof system to prevent bad milk from ever making it into a restaurant meal. N.B. The comparison to Chaos Monkey is the false analogy. People run Chaos Monkey on their own infrastructure. And yes, if you run a tool like that on someone else's infrastructure without their permission, you're the asshole.
- blagie 3y ago
  You don't get it. The core issue here is _systemic risk_ and _systemic resiliency_. The risk profile of driving does not change. By driving a car, I expect a risk of 1.5 deaths per 100 million miles driven. We all agree that's a reasonable risk profile. On the other hand, rare events are things like natural disasters, wars, plaques, asteroids hitting the planet, and so on. Day-to-day decision-making does very poorly for preparing us for those risks. > And yes, if you run a tool like that on someone else's infrastructure without their permission, you're the asshole. No. This is wrong. Things like bug bounty programs were created precisely because whitehat and grayhat hackers made us more resistant to blackhat hackers. If someone tries to compromise my bank's infrastructure _with the intent of surfacing vulnerabilities and without intent of stealing my money_, that might be bad for quarterly profits and my bank might not like it, but it's _good for me_. Free markets push towards low resiliency by offloading risks onto consumers. Behaviors like this change market dynamics in positive ways.
  - ethanbond 3y ago
    Ah got it. So if someone pulls up alongside you at 80mph on a highway and remotely disables your vehicle without your prior consent and you crash into a barricade and your entire family dies, we can just take note of the threat vector and find comfort in the fact that maybe an auto manufacturer will harden their cars against future risks. Makes a ton of sense. > bug bounty programs Who decides whether to operate a bug bounty program and the parameters of said program?
    - blagie 3y ago
      > Ah got it Apparently not.... > So if someone pulls up alongside you at 80mph on a highway and remotely disables your vehicle without your prior consent and you crash into a barricade and your entire family dies, we can just take note of the threat vector and find comfort in the fact that maybe an auto manufacturer will harden their cars against future risks. Makes a ton of sense. A better example: Someone casually remotely disables all cars in parking lots. They need to be unbricked, which is a month-long process. The outcome is cars are built with security built in. That prevents a terrorist attack six months later where someone was plotting to disable thousands of cars on highways in the way you described. Cost: Hundreds of parking lot vehicles disabled, with a range of (very real) consequences, such as missing job interviews, missing school pickups, missed medical appointments, etc. Upside: Dozens of lives saved (but since this never happens, it's abstract lives) It's very much the trolley problem. It's of course possible to set up absurd trolley problems (pull a lever, kill 5 people instead of 1, rather than the other way around, as you're doing). People have different answers to the trolley problem.

by blagie 3y ago

ethanbond 3y ago

It's not a false analogy at all. You just happen to care about cyber risk and resilience, so you think it's okay to accept a higher cost in pursuit of it. Here you go: * You need steel security doors because we should be resilient to crises and broad security threats. You don't know when there will be a major gang war, a neighbor with a psychotic break, a government breakdown, or a ground invasion. It's just better to have steel security doors because there are all sorts of real risks that they mitigate. * You need to drive slowly because some day someone will pull out in front of you, even by accident! Accidents like that happen hundreds or thousands of times per day, so it's frankly insane not to be prepared for it. * You need to check all your restaurant's ingredients because someone will forget to throw out the milk one day. We need to be resilient to this because it's certainly going to happen on occasion, and the right way to achieve resilience is to shift that burden onto end users, or at least to continually poison end users until they demand action from food regulators that we have a 100% foolproof system to prevent bad milk from ever making it into a restaurant meal. N.B. The comparison to Chaos Monkey is the false analogy. People run Chaos Monkey on their own infrastructure. And yes, if you run a tool like that on someone else's infrastructure without their permission, you're the asshole.

blagie 3y ago
You don't get it. The core issue here is _systemic risk_ and _systemic resiliency_. The risk profile of driving does not change. By driving a car, I expect a risk of 1.5 deaths per 100 million miles driven. We all agree that's a reasonable risk profile. On the other hand, rare events are things like natural disasters, wars, plaques, asteroids hitting the planet, and so on. Day-to-day decision-making does very poorly for preparing us for those risks. > And yes, if you run a tool like that on someone else's infrastructure without their permission, you're the asshole. No. This is wrong. Things like bug bounty programs were created precisely because whitehat and grayhat hackers made us more resistant to blackhat hackers. If someone tries to compromise my bank's infrastructure _with the intent of surfacing vulnerabilities and without intent of stealing my money_, that might be bad for quarterly profits and my bank might not like it, but it's _good for me_. Free markets push towards low resiliency by offloading risks onto consumers. Behaviors like this change market dynamics in positive ways.
- ethanbond 3y ago
  Ah got it. So if someone pulls up alongside you at 80mph on a highway and remotely disables your vehicle without your prior consent and you crash into a barricade and your entire family dies, we can just take note of the threat vector and find comfort in the fact that maybe an auto manufacturer will harden their cars against future risks. Makes a ton of sense. > bug bounty programs Who decides whether to operate a bug bounty program and the parameters of said program?
  - blagie 3y ago
    > Ah got it Apparently not.... > So if someone pulls up alongside you at 80mph on a highway and remotely disables your vehicle without your prior consent and you crash into a barricade and your entire family dies, we can just take note of the threat vector and find comfort in the fact that maybe an auto manufacturer will harden their cars against future risks. Makes a ton of sense. A better example: Someone casually remotely disables all cars in parking lots. They need to be unbricked, which is a month-long process. The outcome is cars are built with security built in. That prevents a terrorist attack six months later where someone was plotting to disable thousands of cars on highways in the way you described. Cost: Hundreds of parking lot vehicles disabled, with a range of (very real) consequences, such as missing job interviews, missing school pickups, missed medical appointments, etc. Upside: Dozens of lives saved (but since this never happens, it's abstract lives) It's very much the trolley problem. It's of course possible to set up absurd trolley problems (pull a lever, kill 5 people instead of 1, rather than the other way around, as you're doing). People have different answers to the trolley problem.