3 ms·

The goalpost isn't moving. Comparisons to physical products are wrong because the question is not "are there safety critical bugs" but "can this product survive

The goalpost isn't moving. Comparisons to physical products are wrong because the question is not "are there safety critical bugs" but "can this product survive a sustained assault by highly trained adversaries", which is a standard no other product is held to.

pjmlp 2y ago
Silly me thinking there is something like product recalls, food poisoning, and use cases that insurances don't cover due to high liability in hazardous goods. Anything that brings proper engineering practices into computing, and liability for malpractices, gets my vote.
- mike_hearn 2y ago
  Software bugs almost never poison you, so that isn't applicable. Product recalls happen every day in the software industry, voluntarily. We call them software updates and the industry does them far better and more smoothly than any other industry. Use cases that aren't covered due to liability can be found in any EULA. You aren't allowed to use Windows in nuclear power stations for example.
  - pjmlp 2y ago
    Actually they do, when faulty software powers something food related, anywhere on the delivery chain from the farmer to the point someone actually eats something. EULAs are meaningless in many countries outside US law, most folks are yet to bother to sue companies because they have been badly educated to accept faulty software, while they on material world, that is something that they only accept on cheap knockoffs on street bazaar, and 1 euro shops. Maybe that is the measure for software then. And even those do require a permit for selling, in many countries, or some exchange of goods between sellers and law enforcement checking permits.
Veserv 2y ago
Is a non-network connected car model at risk of all cars being taken over by a economical sustained assault by highly trained adversaries in a foreign country? No. Is a network connected car vulnerable to such attacks? Yes. The product decision to make the car network connected introduced a new, previously non-existent risk. If you want to introduce that new risk, you (and anybody else who wants to introduce such risk) are responsible for the consequences. You can just not do it if that scares you. “But other people get to profit off of endangering society and I want to do it too but I do not want to be responsible for the consequences” is not a very compelling argument for letting you do that. That you need to reach higher standards for new functionality you want to add is the entire point. It is how it works in every other industry. Demanding lower standards because you want to do things that are harder and more dangerous is ass backwards.
- thayne 2y ago
  Ok, now consider that there is a software component, say an open source library, that has a vulnerability in it, and a car maker used that component in their network connected car, then a well funded foreign adversary exploited that vulnerability, among others, to do something nefarious. That software component wasn't specifically designed to be used in cars, but it wasn't specifically designed not to be either, it was a general purpose tool. Should the make of that component be held liable? They weren't the ones who decided to connect a car to the internet. They certainly weren't the ones who decided not to fully isolate the network-connected components from anything that could control the care. But if software doesn't have a way to push liability onto the user, then you can bet the big auto maker will sue that developer in nebraska[1] into oblivion, and the world might be worse off because of it. I don't think that you should be able to do something like sell network connected cars, without putting any effort into security. But at the same time, I think a blanket requirement to be liable for any security vulnerabilities in your software could have a lot of negative consequences. [1]: https://xkcd.com/2347/ https://xkcd.com/2347/
  - pjmlp 2y ago
    If it was an electronic component that would set the car on fire due to a short circuit, the maker of the electronic component would certainly be sued, and was the electronic component designed to be placed on car circuit boards in first place?

pjmlp 2y ago

Silly me thinking there is something like product recalls, food poisoning, and use cases that insurances don't cover due to high liability in hazardous goods. Anything that brings proper engineering practices into computing, and liability for malpractices, gets my vote.

mike_hearn 2y ago
Software bugs almost never poison you, so that isn't applicable. Product recalls happen every day in the software industry, voluntarily. We call them software updates and the industry does them far better and more smoothly than any other industry. Use cases that aren't covered due to liability can be found in any EULA. You aren't allowed to use Windows in nuclear power stations for example.
- pjmlp 2y ago
  Actually they do, when faulty software powers something food related, anywhere on the delivery chain from the farmer to the point someone actually eats something. EULAs are meaningless in many countries outside US law, most folks are yet to bother to sue companies because they have been badly educated to accept faulty software, while they on material world, that is something that they only accept on cheap knockoffs on street bazaar, and 1 euro shops. Maybe that is the measure for software then. And even those do require a permit for selling, in many countries, or some exchange of goods between sellers and law enforcement checking permits.

Veserv 2y ago

Is a non-network connected car model at risk of all cars being taken over by a economical sustained assault by highly trained adversaries in a foreign country? No. Is a network connected car vulnerable to such attacks? Yes. The product decision to make the car network connected introduced a new, previously non-existent risk. If you want to introduce that new risk, you (and anybody else who wants to introduce such risk) are responsible for the consequences. You can just not do it if that scares you. “But other people get to profit off of endangering society and I want to do it too but I do not want to be responsible for the consequences” is not a very compelling argument for letting you do that. That you need to reach higher standards for new functionality you want to add is the entire point. It is how it works in every other industry. Demanding lower standards because you want to do things that are harder and more dangerous is ass backwards.

thayne 2y ago
Ok, now consider that there is a software component, say an open source library, that has a vulnerability in it, and a car maker used that component in their network connected car, then a well funded foreign adversary exploited that vulnerability, among others, to do something nefarious. That software component wasn't specifically designed to be used in cars, but it wasn't specifically designed not to be either, it was a general purpose tool. Should the make of that component be held liable? They weren't the ones who decided to connect a car to the internet. They certainly weren't the ones who decided not to fully isolate the network-connected components from anything that could control the care. But if software doesn't have a way to push liability onto the user, then you can bet the big auto maker will sue that developer in nebraska[1] into oblivion, and the world might be worse off because of it. I don't think that you should be able to do something like sell network connected cars, without putting any effort into security. But at the same time, I think a blanket requirement to be liable for any security vulnerabilities in your software could have a lot of negative consequences. [1]: https://xkcd.com/2347/ https://xkcd.com/2347/
- pjmlp 2y ago
  If it was an electronic component that would set the car on fire due to a short circuit, the maker of the electronic component would certainly be sued, and was the electronic component designed to be placed on car circuit boards in first place?