4 ms·

The threat model of code is different than that of bridges, and legislation won't help. Civil engineers have to make sure that bridges won't naturally collapse

The threat model of code is different than that of bridges, and legislation won't help. Civil engineers have to make sure that bridges won't naturally collapse. Software "engineers" have to ensure that no one will break their code, from the privacy and anonymity of their house, with millions of dollars in payoffs. If bridges held millions of dollars and would be accessible from around the world, they would be breaking all the time also.

mdpopescu 10y ago
Agreed. "Programming Satan's Computer" is a good paper on the subject of code running under constant attack.
nine_k 10y ago
Compare it to automotive engineering. Cars are designed with an assumption that they will be sometimes mishandled, tipped over, crashed into things, run into bodies of water, set on fire, etc. In these circumstances cars should behave as to best preserve humans' safety, both inside (passengers) and outside (pedestrians). This leads to some serious design decisions on deeper level, e.g. making the engine go under the rest of the car on a frontal collision, adding a structural cage around the passenger compartment, etc. Deeply defensive programming methodologies also exist, from using safer languages and formally proven algorithms to pen-testing the completed setup. Their adoption just costs effort and time, that is, money.
- dispose13432 10y ago
  >Cars are designed with an assumption that they will be sometimes mishandled But not maliciously. If you run your car at another, the other car will break. I would compare it to lock-making. Despite thousands of years of lock-making, they _still_ get broken, and there's nothing to be done about it. >Deeply defensive programming methodologies also exist, from using safer languages and formally proven algorithms to pen-testing the completed setup. Their adoption just costs effort and time, that is, money. It's not just money. It's paperwork, bureaucracy, and ultimately not effective. Practically, the only way to ensure safety is to take down the internet.
  - nine_k 10y ago
    Maliciously, too: see all these little things from door locks to built-in speed limiters. Formal certification may be paperwork. But things like using e.g. Haskell instead of Ruby, or Rust instead of C, generating exhaustive tests where possible, making sure that tests touch every line of the code base, writing short pure functions for the most part, and isolating effectful and unsafe code are not about paperwork, to my mind.
    - dispose13432 10y ago
      >Formal certification may be paperwork. But things like using e.g. Haskell instead of Ruby, or Rust instead of C, generating exhaustive tests where possible, making sure that tests touch every line of the code base, writing short pure functions for the most part, and isolating effectful and unsafe code are not about paperwork, to my mind. But you can't legislate languages (C is banned). You'd legislate something along the lines of three hundred pages of legalese requirements which read like Patent applications (there's a reason they look that way) which _would_ require use of a memory safe language but would require that the compiler be signed off by a bunch of lawyers, closed source and costing about $10,000 a license. We had a government language. It was called Ada. There's a reason it never took off.
  - sedachv 10y ago
    > It's not just money. It's paperwork, bureaucracy, and ultimately not effective. Practically, the only way to ensure safety is to take down the internet. https://logfall.wordpress.com/all-or-nothing-fallacy/ https://logfall.wordpress.com/all-or-nothing-fallacy/ We can and should be making incremental improvements to technology. Continuing to use methods that are known to be faulty is gross negligence in the legal sense: https://en.wikipedia.org/wiki/Gross_negligence https://en.wikipedia.org/wiki/Gross_negligence We do not see enough lawsuits around software negligence today because companies are under no obligation to release source code - there is no way to tell how or why or even whether software is defective. This was a major issue in the investigation of the Toyota unintended acceleration scandal. The investigation uncovered major, obvious code quality problems, ultimately concluded the fault was caused by software (something Toyota denied, and without the investigation, consumers never would have learned), and I hope becomes a precedent: http://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code http://www.safetyresearch.net/blog/articles/toyota-unintende... IMO if poor coding practice is gross negligence, it follows that failing to reveal the source code constitutes intent: poorly developed closed-source software is criminal negligence.
hackuser 10y ago
Are you saying it's impossible to write quality code? There is high-assurance software these days.

mdpopescu 10y ago

Agreed. "Programming Satan's Computer" is a good paper on the subject of code running under constant attack.

nine_k 10y ago

Compare it to automotive engineering. Cars are designed with an assumption that they will be sometimes mishandled, tipped over, crashed into things, run into bodies of water, set on fire, etc. In these circumstances cars should behave as to best preserve humans' safety, both inside (passengers) and outside (pedestrians). This leads to some serious design decisions on deeper level, e.g. making the engine go under the rest of the car on a frontal collision, adding a structural cage around the passenger compartment, etc. Deeply defensive programming methodologies also exist, from using safer languages and formally proven algorithms to pen-testing the completed setup. Their adoption just costs effort and time, that is, money.

dispose13432 10y ago
>Cars are designed with an assumption that they will be sometimes mishandled But not maliciously. If you run your car at another, the other car will break. I would compare it to lock-making. Despite thousands of years of lock-making, they _still_ get broken, and there's nothing to be done about it. >Deeply defensive programming methodologies also exist, from using safer languages and formally proven algorithms to pen-testing the completed setup. Their adoption just costs effort and time, that is, money. It's not just money. It's paperwork, bureaucracy, and ultimately not effective. Practically, the only way to ensure safety is to take down the internet.
- nine_k 10y ago
  Maliciously, too: see all these little things from door locks to built-in speed limiters. Formal certification may be paperwork. But things like using e.g. Haskell instead of Ruby, or Rust instead of C, generating exhaustive tests where possible, making sure that tests touch every line of the code base, writing short pure functions for the most part, and isolating effectful and unsafe code are not about paperwork, to my mind.
  - dispose13432 10y ago
    >Formal certification may be paperwork. But things like using e.g. Haskell instead of Ruby, or Rust instead of C, generating exhaustive tests where possible, making sure that tests touch every line of the code base, writing short pure functions for the most part, and isolating effectful and unsafe code are not about paperwork, to my mind. But you can't legislate languages (C is banned). You'd legislate something along the lines of three hundred pages of legalese requirements which read like Patent applications (there's a reason they look that way) which _would_ require use of a memory safe language but would require that the compiler be signed off by a bunch of lawyers, closed source and costing about $10,000 a license. We had a government language. It was called Ada. There's a reason it never took off.
- sedachv 10y ago
  > It's not just money. It's paperwork, bureaucracy, and ultimately not effective. Practically, the only way to ensure safety is to take down the internet. https://logfall.wordpress.com/all-or-nothing-fallacy/ https://logfall.wordpress.com/all-or-nothing-fallacy/ We can and should be making incremental improvements to technology. Continuing to use methods that are known to be faulty is gross negligence in the legal sense: https://en.wikipedia.org/wiki/Gross_negligence https://en.wikipedia.org/wiki/Gross_negligence We do not see enough lawsuits around software negligence today because companies are under no obligation to release source code - there is no way to tell how or why or even whether software is defective. This was a major issue in the investigation of the Toyota unintended acceleration scandal. The investigation uncovered major, obvious code quality problems, ultimately concluded the fault was caused by software (something Toyota denied, and without the investigation, consumers never would have learned), and I hope becomes a precedent: http://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code http://www.safetyresearch.net/blog/articles/toyota-unintende... IMO if poor coding practice is gross negligence, it follows that failing to reveal the source code constitutes intent: poorly developed closed-source software is criminal negligence.

hackuser 10y ago

Are you saying it's impossible to write quality code? There is high-assurance software these days.