7 ms·

Class Breaks

189 points by thegeomaster 10y ago

yuchi 10y ago
Very interesting read. My personal highlight is (about the type of security we are moving into): > It's a world where driverless cars are much safer than people-driven cars, until suddenly they're not.
- zevets 10y ago
  It also means any political figure who "dies in a self driving car accident" will instantly become part of a conspiracy theory.
  - snerbles 10y ago
    The case of Michael Hastings comes to mind. http://nymag.com/news/features/michael-hastings-2013-11/ http://nymag.com/news/features/michael-hastings-2013-11/
golemotron 10y ago
This seems related to the concept of Software Monoculture that Bruce wrote about here: https://www.schneier.com/blog/archives/2010/12/software_monocu.html https://www.schneier.com/blog/archives/2010/12/software_mono... It's a clear case for technological diversity. The hard problem is that economies of scale work against technological diversity. It's similar to what happens with monoculture in biology.
- kmicklas 10y ago
  It's a catch 22 because diversity reduces the potential for class breaks but also increases the probability that any given system is insecure in the first place. (Less resources can be devoted to securing each unique system.)
  - golemotron 10y ago
    I don't think that necessarily follows. People deciding to make a new system with different security would factor that into the cost of the system. It's just as likely to result in fewer systems due to increased cost.
    - kmicklas 10y ago
      That's basically my point. The level of tech diversity is set by some kind of equilibrium between the cost of writing new code and the value of doing so.
matt_wulfeck 10y ago
If we take a cue from nature, it means diversified genetics. This means different IoT manufacturers becoming as snowflaked as possible to avoid diseases that affects other systems. When one system gets too large it only takes a single virus to bring it down. But this runs counter to the "don't roll your own crypto" philosophy. Crypto in this context can be any sort of important aspect of your device that you want to secure from hackers. Right now it's a bad idea to do things on your own, but as systems get even larger and interconnected it may be your only defense.
- rm_-rf_slash 10y ago
  Maybe a better metaphor would be bodily responses to ailments. We usually think of immune systems as white cell warriors but the body has several ways to fight off disease. Fevers are often created by the body to "burn" the offending pathogen from functioning properly and infecting further. Diarrhea is an attempt to purge the body of the disease from the areas it thrives in most. So in this case, roll your own crypto and use known strong crypto. Don't rely on either on their own. Example: use end-to-end encryption apps like Signal, but if necessary, communicate in coded language that only the person on the other end would understand.
- mi100hael 10y ago
  I think you're missing the fact that a line of devices from a manufacturer is still a "class." If you have thousands or millions of devices out there with un-tested, DIY crypto, then that's still thousands or millions of devices rendered malicious when a vulnerability is found & exploited. At least vulnerabilities like Heartbleed were quickly patched and widely-publicized. IoT vendors have been historically horrible when it comes to providing updates.
tptacek 10y ago
For what it's worth, the more idiomatic term in computer security is "bug class". I know "class break" is potentially the better, more general term, but nobody I talk to ever really uses it.
- wglb 10y ago
  In the insurance industry it is called Category Risk. As in if you insure houses instead o cars you get lots of exposure with a hurricane.
rm_-rf_slash 10y ago
I'm surprised this isn't getting much attention here. There really isn't a technological solution for class breaks. Go big and you replace one kind of class break with another (a top-tier security system for hotel card readers just focuses the attacks on that system). Go small and it isn't cost effective or plausible at scale (everybody install gentoo! Compile your own kernels!). Honestly, I'm stumped. We rely on these systems more and more to squeeze out every bit of productivity we can get, but that only leaves us at the mercy of our systems. Is it even possible to fix this kind of situation before it really turns on us, or are we doomed to suffer a cataclysm we cannot yet imagine?
- irq-1 10y ago
  As he said, we do have some experience with Class Breaks. Off the top of my head: Palm Pilots had a little button on the back to reset them. Chrome Books have a read-only OS/boot loader. Old software licensing dongles covered the chips with epoxy. We use VLANS to separate classes of traffic. x86 uses rings to separate classes of code. Firewalls and QOS systems separate classes of traffic.
- wyldfire 10y ago
  The cure is worse than the disease. But it's coming. It will come as a devastating one-two punch: the end of general purpose computing devices and the rise of a publicly-identified Internet. The former will be phased in as countries legislate it. Legislators won't be concerned with the slippery slope between abacus and iphone, the courts will likely draw a distinction somewhere around anything north of "silicon-based electronics with the right resources for an IP stack". A few countries will be bastions of Free Computing, but many/most will draw up similar laws. The latter will be held back by Metcalfe's Law until some near-global momentum can be built. It would likely take a simultaneous attack on many global corporations/political parties/beloved individuals, but it seems like that's just a matter of time. Faced with repeated attacks, the public won't accept "but we designed it that way intentionally" as valid.
  - zevets 10y ago
    Abandoning general purpose computing is too crazy, as it amounts to throwing away software. The anonymous internet is a dead-man walking, but silicon valley is as complicit in this as the government will be. However, software engineering will change into a much more regulated environment. Writing software for the web will eventually become like writing software for the space shuttle: slow, rigorous and very expensive. It will also kill the hacker culture, as hacking together a piece of software will become like hacking together a bridge: illegal.
ChicagoBoy11 10y ago
I can't help but think this sort of issue far outweighs whatever concern we may have when it comes to any sort of bad AI actor. Every time I read about the OTA updates on Tesla cars my spine tingles a bit about the possibility (which I can only conceive is greater than 0) of some bad actor sending out code which turns those cars into ungoverned highway missiles. If there's one lesson that we can take from all the data breaches from top companies and high-profile government officials is that we are terrible at securing software. The 21st century Ted Kaczyinski isn't going to need the post office; he's got a MBP and SSH.
- gist 10y ago
  > of some bad actor sending out code which turns those cars into ungoverned highway missiles Yes but think of the probabilities that have to happen are: a) Knowledgeable person b) Motivated person c) Evil or mentally unstable person d) Determination and ability to overcome adversity. And the person or person who has to decide that Tesla is what they want to exploit instead of another car manufacturer or another object and so on. Compare this with someone who simply wants to take their car and drive it into a crowd.
  - ysavir 10y ago
    If we're assuming independent hackers that simply have nothing better to do... Simply look at how many hackers do it already. If we're assuming hackers that realize they can lock down a person's car in exchange for ransom... Simply look at how many hackers do it already. If we're assuming a broad attack by a foreign entity... Simply look at how many hackers do it already.
  - mlgh 10y ago
    9/11
  - aeykie 10y ago
    How many times can one individual crash their car into a crowd vs force a hacked car into a crowd though?
  - jimbokun 10y ago
jimmytidey 10y ago
Very similar argument to lovely CPG Grey video: https://www.youtube.com/watch?v=VPBH1eW28mo https://www.youtube.com/watch?v=VPBH1eW28mo
Coincoin 10y ago
This is exactly why I hate electronic voting. When I argue about it, people always tell me that it's easier to commit vote fraud with paper ballots. Sure, it's easier for a single person to temper with a single paper ballot or even poll box, but it is much harder on a large scale. Tampering with every physical boxes and tallies in a country requires much more effort and organisation than exploiting a single vulnerability that applies to an endless series of identical machines.
- gertef 10y ago
  https://en.wikipedia.org/wiki/Voter-verified_paper_audit_trail https://en.wikipedia.org/wiki/Voter-verified_paper_audit_tra...
- kmicklas 10y ago
  Advances in mathematically verified software could help here - although mechanisms for verifying the manufacturing of chips all the way down the supply chain might be farther out.
- taeric 10y ago
  The biggest actual risk to voting remains the party in power. Why go through the effort of fraud, when you can just throw out the results? Similarly, propaganda is also more effective.
  - abecedarius 10y ago
    Why hide that you're changing the election result? Well, I can imagine cases where it's not worth the bother, but they hardly seem like the default.
    - taeric 10y ago
      My point is look at pretty much every election where democracy actually left. Not exactly subtle fraud going on. (Though, I will confess I have not tabulated this, so would be very curious in full numbers.)
taeric 10y ago
A better analogy would be crops and the dangers of a mono culture. A class break is akin to a disease that an entire crop is susceptible to. See, e.g., the bole weavil. Or the reason the current popular banana is at risk.

189 points by thegeomaster 10y ago

yuchi 10y ago

Very interesting read. My personal highlight is (about the type of security we are moving into): > It's a world where driverless cars are much safer than people-driven cars, until suddenly they're not.

zevets 10y ago
It also means any political figure who "dies in a self driving car accident" will instantly become part of a conspiracy theory.
- snerbles 10y ago
  The case of Michael Hastings comes to mind. http://nymag.com/news/features/michael-hastings-2013-11/ http://nymag.com/news/features/michael-hastings-2013-11/

golemotron 10y ago

This seems related to the concept of Software Monoculture that Bruce wrote about here: https://www.schneier.com/blog/archives/2010/12/software_monocu.html https://www.schneier.com/blog/archives/2010/12/software_mono... It's a clear case for technological diversity. The hard problem is that economies of scale work against technological diversity. It's similar to what happens with monoculture in biology.

kmicklas 10y ago
It's a catch 22 because diversity reduces the potential for class breaks but also increases the probability that any given system is insecure in the first place. (Less resources can be devoted to securing each unique system.)
- golemotron 10y ago
  I don't think that necessarily follows. People deciding to make a new system with different security would factor that into the cost of the system. It's just as likely to result in fewer systems due to increased cost.
  - kmicklas 10y ago
    That's basically my point. The level of tech diversity is set by some kind of equilibrium between the cost of writing new code and the value of doing so.

matt_wulfeck 10y ago

If we take a cue from nature, it means diversified genetics. This means different IoT manufacturers becoming as snowflaked as possible to avoid diseases that affects other systems. When one system gets too large it only takes a single virus to bring it down. But this runs counter to the "don't roll your own crypto" philosophy. Crypto in this context can be any sort of important aspect of your device that you want to secure from hackers. Right now it's a bad idea to do things on your own, but as systems get even larger and interconnected it may be your only defense.

rm_-rf_slash 10y ago
Maybe a better metaphor would be bodily responses to ailments. We usually think of immune systems as white cell warriors but the body has several ways to fight off disease. Fevers are often created by the body to "burn" the offending pathogen from functioning properly and infecting further. Diarrhea is an attempt to purge the body of the disease from the areas it thrives in most. So in this case, roll your own crypto and use known strong crypto. Don't rely on either on their own. Example: use end-to-end encryption apps like Signal, but if necessary, communicate in coded language that only the person on the other end would understand.
mi100hael 10y ago
I think you're missing the fact that a line of devices from a manufacturer is still a "class." If you have thousands or millions of devices out there with un-tested, DIY crypto, then that's still thousands or millions of devices rendered malicious when a vulnerability is found & exploited. At least vulnerabilities like Heartbleed were quickly patched and widely-publicized. IoT vendors have been historically horrible when it comes to providing updates.

tptacek 10y ago

For what it's worth, the more idiomatic term in computer security is "bug class". I know "class break" is potentially the better, more general term, but nobody I talk to ever really uses it.

wglb 10y ago
In the insurance industry it is called Category Risk. As in if you insure houses instead o cars you get lots of exposure with a hurricane.

rm_-rf_slash 10y ago

I'm surprised this isn't getting much attention here. There really isn't a technological solution for class breaks. Go big and you replace one kind of class break with another (a top-tier security system for hotel card readers just focuses the attacks on that system). Go small and it isn't cost effective or plausible at scale (everybody install gentoo! Compile your own kernels!). Honestly, I'm stumped. We rely on these systems more and more to squeeze out every bit of productivity we can get, but that only leaves us at the mercy of our systems. Is it even possible to fix this kind of situation before it really turns on us, or are we doomed to suffer a cataclysm we cannot yet imagine?

irq-1 10y ago
As he said, we do have some experience with Class Breaks. Off the top of my head: Palm Pilots had a little button on the back to reset them. Chrome Books have a read-only OS/boot loader. Old software licensing dongles covered the chips with epoxy. We use VLANS to separate classes of traffic. x86 uses rings to separate classes of code. Firewalls and QOS systems separate classes of traffic.
wyldfire 10y ago
The cure is worse than the disease. But it's coming. It will come as a devastating one-two punch: the end of general purpose computing devices and the rise of a publicly-identified Internet. The former will be phased in as countries legislate it. Legislators won't be concerned with the slippery slope between abacus and iphone, the courts will likely draw a distinction somewhere around anything north of "silicon-based electronics with the right resources for an IP stack". A few countries will be bastions of Free Computing, but many/most will draw up similar laws. The latter will be held back by Metcalfe's Law until some near-global momentum can be built. It would likely take a simultaneous attack on many global corporations/political parties/beloved individuals, but it seems like that's just a matter of time. Faced with repeated attacks, the public won't accept "but we designed it that way intentionally" as valid.
- zevets 10y ago
  Abandoning general purpose computing is too crazy, as it amounts to throwing away software. The anonymous internet is a dead-man walking, but silicon valley is as complicit in this as the government will be. However, software engineering will change into a much more regulated environment. Writing software for the web will eventually become like writing software for the space shuttle: slow, rigorous and very expensive. It will also kill the hacker culture, as hacking together a piece of software will become like hacking together a bridge: illegal.

ChicagoBoy11 10y ago

I can't help but think this sort of issue far outweighs whatever concern we may have when it comes to any sort of bad AI actor. Every time I read about the OTA updates on Tesla cars my spine tingles a bit about the possibility (which I can only conceive is greater than 0) of some bad actor sending out code which turns those cars into ungoverned highway missiles. If there's one lesson that we can take from all the data breaches from top companies and high-profile government officials is that we are terrible at securing software. The 21st century Ted Kaczyinski isn't going to need the post office; he's got a MBP and SSH.

gist 10y ago
> of some bad actor sending out code which turns those cars into ungoverned highway missiles Yes but think of the probabilities that have to happen are: a) Knowledgeable person b) Motivated person c) Evil or mentally unstable person d) Determination and ability to overcome adversity. And the person or person who has to decide that Tesla is what they want to exploit instead of another car manufacturer or another object and so on. Compare this with someone who simply wants to take their car and drive it into a crowd.
- ysavir 10y ago
  If we're assuming independent hackers that simply have nothing better to do... Simply look at how many hackers do it already. If we're assuming hackers that realize they can lock down a person's car in exchange for ransom... Simply look at how many hackers do it already. If we're assuming a broad attack by a foreign entity... Simply look at how many hackers do it already.
- mlgh 10y ago
  9/11
- aeykie 10y ago
  How many times can one individual crash their car into a crowd vs force a hacked car into a crowd though?
- jimbokun 10y ago

jimmytidey 10y ago

Very similar argument to lovely CPG Grey video: https://www.youtube.com/watch?v=VPBH1eW28mo https://www.youtube.com/watch?v=VPBH1eW28mo

Coincoin 10y ago

This is exactly why I hate electronic voting. When I argue about it, people always tell me that it's easier to commit vote fraud with paper ballots. Sure, it's easier for a single person to temper with a single paper ballot or even poll box, but it is much harder on a large scale. Tampering with every physical boxes and tallies in a country requires much more effort and organisation than exploiting a single vulnerability that applies to an endless series of identical machines.

gertef 10y ago
https://en.wikipedia.org/wiki/Voter-verified_paper_audit_trail https://en.wikipedia.org/wiki/Voter-verified_paper_audit_tra...
kmicklas 10y ago
Advances in mathematically verified software could help here - although mechanisms for verifying the manufacturing of chips all the way down the supply chain might be farther out.
taeric 10y ago
The biggest actual risk to voting remains the party in power. Why go through the effort of fraud, when you can just throw out the results? Similarly, propaganda is also more effective.
- abecedarius 10y ago
  Why hide that you're changing the election result? Well, I can imagine cases where it's not worth the bother, but they hardly seem like the default.
  - taeric 10y ago
    My point is look at pretty much every election where democracy actually left. Not exactly subtle fraud going on. (Though, I will confess I have not tabulated this, so would be very curious in full numbers.)

taeric 10y ago

A better analogy would be crops and the dangers of a mono culture. A class break is akin to a disease that an entire crop is susceptible to. See, e.g., the bole weavil. Or the reason the current popular banana is at risk.