5 ms·

This is what holds me back from 'smart' devices that have the potential to cause real harm... We've been making motors (electric or combustion) for over a hund

This is what holds me back from 'smart' devices that have the potential to cause real harm... We've been making motors (electric or combustion) for over a hundred years, and gotten pretty damn good at making them safe and reliable. Same thing with stoves, HVAC equipment, small appliances, etc. These are all mature technologies that we can practically trust our lives with. Internet-connected smart vehicles aren't a mature technology. Not in the sense of this being the win2k era of that tech, but that our assumptions about how to build these systems might be fundamentally wrong. I don't know if it will ever be safe enough to trust human lives to it. Until then, I'll only want to buy cars made before 2010.

iRomain 6y ago
I think the key here, and unfortunately most companies don't give a sh*t, is to allow the user to gain control over his device and/or take it offline if it pleases him. For example, a Tesla car should come with an option to disable any remote control features, or a way to control it over short distance only when it's offline (I don't know if its already the case, I don't have a tesla).
- judge2020 6y ago
  It's not possible without rooting your car and losing the warranty. This is legal as long as you don't touch the odometer.
  - Reelin 6y ago
    > rooting your car Something has gone terribly wrong with modern technology.
- Silhouette 6y ago
  I think the key here...is to allow the user to gain control over his device and/or take it offline if it pleases him. I wonder how long it will be before we start to see legal or regulatory interventions in this area. Mandatory self-updating and phone-home functionality is rapidly infecting technologies we rely on every day, from our cars to our home computers to our TV sets. This always-connected, always-updated approach inevitably introduces some risks. It often causes intrusions into privacy or brings changes after purchase that users of these technologies don't necessarily want. Competition in these markets is evidently insufficient to provide alternatives for those who don't want anything to do with this modern culture. I don't believe that is limited to a small group of eccentric tinfoil-hat fans any more. Hopefully it won't take some sort of widespread disaster to wake the politicians up to the dangers here, though given the past performance of the political class around the world when it comes to technology issues, I'm not particularly optimistic.
  - CamperBob2 6y ago
    Rest assured, the 'regulatory interference' will be in the direction of forbidding the user to gain control over his device and/or take it offline. Consumers don't have a voice here, and (given the Democrats' record) I don't see that changing regardless of who wins the upcoming election.
    - Silhouette 6y ago
      Fortunately, there is a world outside the US, and much of it is more enlightened. If other large markets impose limitations on this kind of technology or create a penalty regime for failures that makes it worthwhile for the manufacturers to invest in more reliable systems, that will probably benefit everyone indirectly, even users in the US if a US government of whatever colour sells them out.
  - donmcronald 6y ago
    > I wonder how long it will be before we start to see legal or regulatory interventions in this area. Not long IMO. It'll be sold as a safety measure, but the real purpose will be to limit competition. I think it's similar to "warranty void if removed" stickers, but I'm worried we're not going to get the same pragmatic legislation that makes those ignorable.
- angry_octet 6y ago
  If you look at adversarial machine learning you'll see it is shaping up as a bit of an evolutionary battle. It is quite likely that a years-offline Telsa (or similar) could be deceived into a fatal collision. We need safe updates with transparent documentation as to all the changes, with hardware enforced feature switches. Forcing them to go through an approval process doesn't sound bad until you've met regulators like the FAA -- or Apple.
  - adrianN 6y ago
    You have to weigh the risks of manipulating cars one at a time because they don't have the latest updates with the risk of manipulating all cars at once because they can be remote controlled. A more expensive alternative to over the air updates is requiring regular updates done by a mechanic, e.g. when the vehicle is due for an inspection.
    - angry_octet 6y ago
      I don't really think OTA updates is the problem, so much as the automatic application of the updates without owner awareness (or consent to possibly significant feature changes). Preventing OTA is not the same as preventing remote control -- vehicles WILL be networked, if only for collision avoidance. OTA could be very useful if a bad exploit becomes widely available -- like a protocol bug in the V2X network that lets you crash the V2X module. Many bugs are discovered after being in production for years. If you allow users to approve and roll back updates then there is some ability to recover after bad updates. Of course, you don't want theives to be able to roll back to a vulnerable release, but you can require the master owner key & code for rollback.
Jeriko 6y ago
"Internet-connected smart vehicles aren't a mature technology. Not in the sense of this being the win2k era of that tech, but that our assumptions about how to build these systems might be fundamentally wrong. I don't know if it will ever be safe enough to trust human lives to it." I often hear this kind of thing and am really surprised by it. Specifically for the tech in vehicles example, it seems like a real double standard. Around 37,000 people in the US die in car accidents every year[1]. That's 100 people a DAY. There is a huge cost to not adopting new safety measures, even if it depends on immature tech, and that needs to be factored against the potential new unknown risks. Driving to work is almost certainly the riskiest thing you do most days. I find it plausible that people 50 years from now will think that the cars we drove before 2010 were unconscionable death traps. [1] https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
- m463 6y ago
  Maybe it's maturity in the sense of relationships. An immature relationship might be based on selfish interests, compounded with power struggles, boundary issues, jealousy, spying, and a little rage and stockholm syndrome thrown in for fun. A mature relationship is based on trust and mutual respect. It does not concern itself with knowing every detail or trepidation regarding other suitors. The relationship is there because it is valued and there are willing participants. Now... internet connected devices.
- mikepurvis 6y ago
  Hopefully in 50 years the whole idea of single-occupancy vehicles will be considered a quaint relic of a much more decadent time, especially vehicles which were allowed such extravagant externalities in terms of pollution and endangerment to vulnerable road users.
- angry_octet 6y ago
  You're comparing apples and oranges. It can both be true that security (and overall software) quality is poor, and that automated buggy cars are better than erratic people. However, when you are considering system risk (e.g. that a bad actor could crash 100k cars at the same time) the worst case outcome could be much worse than the mean outcome.
userbinator 6y ago
I've noticed a lot of older software engineers seem to avoid anything "smart", and quite a few of them are into vintage cars too. I don't think that's coincidental; my daily driver is approaching 50, and completely lacks any computer or electronics for its main purpose.
- WWLink 6y ago
  I dunno if that means anything. Some of my software engineer friends have Tesla cars and love that kinda stuff. Others have old cars with very little in the way of electronics. Some have both! Gotta remember, that every day people entrust their lives to lots of software and electronic systems. Of course, it's important to have reasonable security measure taken, but just because something's electronic doesn't make that a bad thing.
benhurmarcel 6y ago
> Until then, I'll only want to buy cars made before 2010. Most modern cars don't connect to the internet.

iRomain 6y ago

I think the key here, and unfortunately most companies don't give a sh*t, is to allow the user to gain control over his device and/or take it offline if it pleases him. For example, a Tesla car should come with an option to disable any remote control features, or a way to control it over short distance only when it's offline (I don't know if its already the case, I don't have a tesla).

judge2020 6y ago
It's not possible without rooting your car and losing the warranty. This is legal as long as you don't touch the odometer.
- Reelin 6y ago
  > rooting your car Something has gone terribly wrong with modern technology.
Silhouette 6y ago
I think the key here...is to allow the user to gain control over his device and/or take it offline if it pleases him. I wonder how long it will be before we start to see legal or regulatory interventions in this area. Mandatory self-updating and phone-home functionality is rapidly infecting technologies we rely on every day, from our cars to our home computers to our TV sets. This always-connected, always-updated approach inevitably introduces some risks. It often causes intrusions into privacy or brings changes after purchase that users of these technologies don't necessarily want. Competition in these markets is evidently insufficient to provide alternatives for those who don't want anything to do with this modern culture. I don't believe that is limited to a small group of eccentric tinfoil-hat fans any more. Hopefully it won't take some sort of widespread disaster to wake the politicians up to the dangers here, though given the past performance of the political class around the world when it comes to technology issues, I'm not particularly optimistic.
- CamperBob2 6y ago
  Rest assured, the 'regulatory interference' will be in the direction of forbidding the user to gain control over his device and/or take it offline. Consumers don't have a voice here, and (given the Democrats' record) I don't see that changing regardless of who wins the upcoming election.
  - Silhouette 6y ago
    Fortunately, there is a world outside the US, and much of it is more enlightened. If other large markets impose limitations on this kind of technology or create a penalty regime for failures that makes it worthwhile for the manufacturers to invest in more reliable systems, that will probably benefit everyone indirectly, even users in the US if a US government of whatever colour sells them out.
- donmcronald 6y ago
  > I wonder how long it will be before we start to see legal or regulatory interventions in this area. Not long IMO. It'll be sold as a safety measure, but the real purpose will be to limit competition. I think it's similar to "warranty void if removed" stickers, but I'm worried we're not going to get the same pragmatic legislation that makes those ignorable.
angry_octet 6y ago
If you look at adversarial machine learning you'll see it is shaping up as a bit of an evolutionary battle. It is quite likely that a years-offline Telsa (or similar) could be deceived into a fatal collision. We need safe updates with transparent documentation as to all the changes, with hardware enforced feature switches. Forcing them to go through an approval process doesn't sound bad until you've met regulators like the FAA -- or Apple.
- adrianN 6y ago
  You have to weigh the risks of manipulating cars one at a time because they don't have the latest updates with the risk of manipulating all cars at once because they can be remote controlled. A more expensive alternative to over the air updates is requiring regular updates done by a mechanic, e.g. when the vehicle is due for an inspection.
  - angry_octet 6y ago
    I don't really think OTA updates is the problem, so much as the automatic application of the updates without owner awareness (or consent to possibly significant feature changes). Preventing OTA is not the same as preventing remote control -- vehicles WILL be networked, if only for collision avoidance. OTA could be very useful if a bad exploit becomes widely available -- like a protocol bug in the V2X network that lets you crash the V2X module. Many bugs are discovered after being in production for years. If you allow users to approve and roll back updates then there is some ability to recover after bad updates. Of course, you don't want theives to be able to roll back to a vulnerable release, but you can require the master owner key & code for rollback.

Jeriko 6y ago

"Internet-connected smart vehicles aren't a mature technology. Not in the sense of this being the win2k era of that tech, but that our assumptions about how to build these systems might be fundamentally wrong. I don't know if it will ever be safe enough to trust human lives to it." I often hear this kind of thing and am really surprised by it. Specifically for the tech in vehicles example, it seems like a real double standard. Around 37,000 people in the US die in car accidents every year[1]. That's 100 people a DAY. There is a huge cost to not adopting new safety measures, even if it depends on immature tech, and that needs to be factored against the potential new unknown risks. Driving to work is almost certainly the riskiest thing you do most days. I find it plausible that people 50 years from now will think that the cars we drove before 2010 were unconscionable death traps. [1] https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...

m463 6y ago
Maybe it's maturity in the sense of relationships. An immature relationship might be based on selfish interests, compounded with power struggles, boundary issues, jealousy, spying, and a little rage and stockholm syndrome thrown in for fun. A mature relationship is based on trust and mutual respect. It does not concern itself with knowing every detail or trepidation regarding other suitors. The relationship is there because it is valued and there are willing participants. Now... internet connected devices.
mikepurvis 6y ago
Hopefully in 50 years the whole idea of single-occupancy vehicles will be considered a quaint relic of a much more decadent time, especially vehicles which were allowed such extravagant externalities in terms of pollution and endangerment to vulnerable road users.
angry_octet 6y ago
You're comparing apples and oranges. It can both be true that security (and overall software) quality is poor, and that automated buggy cars are better than erratic people. However, when you are considering system risk (e.g. that a bad actor could crash 100k cars at the same time) the worst case outcome could be much worse than the mean outcome.

userbinator 6y ago

I've noticed a lot of older software engineers seem to avoid anything "smart", and quite a few of them are into vintage cars too. I don't think that's coincidental; my daily driver is approaching 50, and completely lacks any computer or electronics for its main purpose.

WWLink 6y ago
I dunno if that means anything. Some of my software engineer friends have Tesla cars and love that kinda stuff. Others have old cars with very little in the way of electronics. Some have both! Gotta remember, that every day people entrust their lives to lots of software and electronic systems. Of course, it's important to have reasonable security measure taken, but just because something's electronic doesn't make that a bad thing.

benhurmarcel 6y ago

> Until then, I'll only want to buy cars made before 2010. Most modern cars don't connect to the internet.