15 ms·

This is clever, and Tesla's software is awful, but anyone doing this for real, with intent to divert cars from the road etc. would be committing a pretty seriou

This is clever, and Tesla's software is awful, but anyone doing this for real, with intent to divert cars from the road etc. would be committing a pretty serious crime.

Polylactic_acid 6y ago
The wonderful thing about IoT is the criminal can be on the other side of the world with no chance of any justice coming to them.
- falcolas 6y ago
  Not to mention, from TFA: "They're also so fleeting that they're difficult for the human eye to detect, and leave behind very little evidence." We've already got people who do things purely for the lulz (swatting, anybody) who could easily justify away these kinds of activities.
  - mattbee 6y ago
    Both good points - so this kind of crime would likely be indistinguishable from a random, terrifying software failure, with the blame falling on the car maker.
    - hellotomyrars 6y ago
      I mean it is a failing of the manufacturer on some level. Not saying someone intentionally exploiting the car isn't also at fault, but I also don't think it's unrealistic to hold the manufacturer accountable as well.
    - piracy1 6y ago
      Tesla logs so much data and stores it and has used it to defend itself in various legal / PR matters. I wonder if they would catch it if someone did this. Or maybe suddenly they wouldn't have that data lol
    - a_bonobo 6y ago
      Very often I see these tiny articles in local newspapers, 15 sentences, that go something like 'The driver lost control of their vehicle and died in the crash'. I've had some terrifying software problems in Audis, including random shutdowns while driving due to faulty sensors, how often do you think someone who 'lost control of their car', which implies that there was human error, died because their car's electronics or software were faulty? That number definitely is not zero.
      - mattbee 6y ago
        Yeah! I'm sure that's happened in software in less sophisticated, but maybe more understandable ways. Lane-keeping & auto-braking software - which loads of cars have - is absolutely terrifying to think about, and I'd be fascinated to see these attacked from a research POV. My old Celica tried to murder me on the M62 and Toyota denied the problem for years (the gas pedals got stuck under the floor mats). And that was just a really obvious (in retrospect) design flaw - for how long could a car maker viably deny safety problems in their software?
- Buttons840 6y ago
  How long until someone claims one of these images designed to cause trouble for AIs is free speech? And are they right?
  - someguy101010 6y ago
    Id say no to the free speech since that sounds similar to shouting fire in a movie theater
    - dragonwriter 6y ago
      Shouting fire in a movie theater is not a good example of unprotected speech, and as such it's not a good place to start reasoning from in deciding of other speech is unprotected. Otherwise, you start banning core political speech like advocacy against the draft, just like the since-overturned case that bad example comes from.
      - tshaddox 6y ago
        The fact that that example became well-known due to the awful decision in Schenck v. United States does not mean that it's not a good example of speech that ought not be protected. Speech that is false and intentionally causes a panic should not be protected.
        katzgrau 6y ago
        > Speech that is false and intentionally causes a panic should not be protected. What is true or false isn't always clear though, even if there's a popular or official consensus. The point of having a public forum is to give new ideas a chance, as unheard of or unpopular as they may be. Of course, with that you get the risk of misinformation as well. As a society we must choose how much risk we're willing to tolerate, and really consider if the the downsides of the level of free speech we have today are really so much different than what we've historically dealt with.
        6y ago
- TaylorAlexander 6y ago
  The trick is to construct a real looking advertisement that “accidentally” triggers this effect, and order the ad placed on digital billboards in Silicon Valley through a fake company located somewhere far far away.
  - userbinator 6y ago
    There have already been (audio) ads that trigger speech-recognising machines like Alexa, and in that case I would very much say it is free speech.
    - MaxBarraclough 6y ago
      Like the famous 2017 Burger King ad that deliberately hijacked nearby Google Home smart speakers. [0] There's an argument to be made that it should be covered by cyber-attack laws. You aren't allowed to deliberately take control of someone else's computer without their permission. Consider a hypothetical example: an author without use of their hands, who uses text-to-speech to write books and to control their computer. Imagine if they put you on speakerphone and you read out a sequence of commands to delete their work. That certainly wouldn't be covered under freedom of expression. There's an obvious difference in degree, as spamming someone by hijacking their smart speaker isn't all that costly, but it's similar in category. [0] https://en.wikipedia.org/wiki/Burger_King_advertising#2000s%E2%80%932010s https://en.wikipedia.org/wiki/Burger_King_advertising#2000s%...
      - dorgo 6y ago
        >There's an argument to be made that it should be covered by cyber-attack laws. Yes, but smart speakers should not rely on laws to prevent successfull attacks.
- quickthrower2 6y ago
  Low chance of any justice.
- gridlockd 6y ago
  Your run-of-the-mill criminal has some sort of profit motive and this seems like a very roundabout way to attempt extortion. A terrorist on the other hand would probably be dissatisfied with the not-quite-catastophic outcome. If you want to cause real havoc, there are far more valuable targets to hack into.
  - josefx 6y ago
    We have idiots that call swat teams on twitch streamers. The profit is for the lulz.
    - gridlockd 6y ago
      Yes, any idiot can call 911 and have SWAT team dispatched, but that is a very low effort "attack". I'm not saying that this could never possibly happen, but it's not much of a real threat exactly because effort vs reward is unfavorable.
- handol 6y ago
  What about the criminal who mounted the screen next to the road and hooked it up to the internet?
accurrent 6y ago
To be fair I think autonomous vehicles should have a requirement that they come with a RADAR or LIDAR as a backup sensor (even if its a low resolution one). Using ML on images for everything is not going to solve your safety problems cause theres no way to gaurantee the system will not crashed when presented with a fake input. Its a darn sight harder to pretend to be farther away on a radar or lidar though. At the very least the autonomous vehicle should be able to gaurantee you don't crash into static obstacles.
- hellotomyrars 6y ago
  As far as I know most people working in the space still think that way, with the very notable exception of Tesla.
Balgair 6y ago
Why would that be a crime? The driver is ultimately responsible, not the car.
- mattbee 6y ago
  If you know that displaying unofficial highway signs will cause "self-driving" cars to swerve off the road, it's not much different an act from dropping breeze blocks off a motorway bridge. I've no idea whether that'd be a highway crime, a computer misuse crime, or just attempted murder - that'd be a new one for lawyers in every country to argue.
jacobsenscott 6y ago
There are already billboards with pictures of stop signs, stop lights, etc on them. It doesn't take any criminal intent. Things that were once just billboards are suddenly hazards.
- im3w1l 6y ago
  Seems reasonable enough to ban that. Might confuse a tired human too.
  - Ekaros 6y ago
    I think banning Teslas as unsafe is even more reasonable conclusion. Those cars can't be trusted so they should not be allowed on public roads. Problem solved. Same goes for any other car with this issue.
    - 542354234235 6y ago
      Or human drivers for that matter. I heard that they have a very high failure rate and cause over 6,500,000 collisions a year in the US alone. Take them back to the drawing board and work out all the bugs until they can put a safe species of driver on the road.
  - jfim 6y ago
    Or maybe remind them to stop for school buses: https://www.wkyt.com/content/news/New-billboards-to-remind-drivers-to-stop-for-school-buses-436832083.html https://www.wkyt.com/content/news/New-billboards-to-remind-d...
onion2k 6y ago
Maybe Lucid will an advert that deliberately makes Teslas stop in front of it.

This is clever, and Tesla's software is awful, but anyone doing this for real, with intent to divert cars from the road etc. would be committing a pretty serious crime.

Polylactic_acid 6y ago

The wonderful thing about IoT is the criminal can be on the other side of the world with no chance of any justice coming to them.

falcolas 6y ago
Not to mention, from TFA: "They're also so fleeting that they're difficult for the human eye to detect, and leave behind very little evidence." We've already got people who do things purely for the lulz (swatting, anybody) who could easily justify away these kinds of activities.
- mattbee 6y ago
  Both good points - so this kind of crime would likely be indistinguishable from a random, terrifying software failure, with the blame falling on the car maker.
  - hellotomyrars 6y ago
    I mean it is a failing of the manufacturer on some level. Not saying someone intentionally exploiting the car isn't also at fault, but I also don't think it's unrealistic to hold the manufacturer accountable as well.
  - piracy1 6y ago
    Tesla logs so much data and stores it and has used it to defend itself in various legal / PR matters. I wonder if they would catch it if someone did this. Or maybe suddenly they wouldn't have that data lol
  - a_bonobo 6y ago
    Very often I see these tiny articles in local newspapers, 15 sentences, that go something like 'The driver lost control of their vehicle and died in the crash'. I've had some terrifying software problems in Audis, including random shutdowns while driving due to faulty sensors, how often do you think someone who 'lost control of their car', which implies that there was human error, died because their car's electronics or software were faulty? That number definitely is not zero.
    - mattbee 6y ago
      Yeah! I'm sure that's happened in software in less sophisticated, but maybe more understandable ways. Lane-keeping & auto-braking software - which loads of cars have - is absolutely terrifying to think about, and I'd be fascinated to see these attacked from a research POV. My old Celica tried to murder me on the M62 and Toyota denied the problem for years (the gas pedals got stuck under the floor mats). And that was just a really obvious (in retrospect) design flaw - for how long could a car maker viably deny safety problems in their software?
Buttons840 6y ago
How long until someone claims one of these images designed to cause trouble for AIs is free speech? And are they right?
- someguy101010 6y ago
  Id say no to the free speech since that sounds similar to shouting fire in a movie theater
  - dragonwriter 6y ago
    Shouting fire in a movie theater is not a good example of unprotected speech, and as such it's not a good place to start reasoning from in deciding of other speech is unprotected. Otherwise, you start banning core political speech like advocacy against the draft, just like the since-overturned case that bad example comes from.
    - tshaddox 6y ago
      The fact that that example became well-known due to the awful decision in Schenck v. United States does not mean that it's not a good example of speech that ought not be protected. Speech that is false and intentionally causes a panic should not be protected.
      - katzgrau 6y ago
        > Speech that is false and intentionally causes a panic should not be protected. What is true or false isn't always clear though, even if there's a popular or official consensus. The point of having a public forum is to give new ideas a chance, as unheard of or unpopular as they may be. Of course, with that you get the risk of misinformation as well. As a society we must choose how much risk we're willing to tolerate, and really consider if the the downsides of the level of free speech we have today are really so much different than what we've historically dealt with.
        6y ago
TaylorAlexander 6y ago
The trick is to construct a real looking advertisement that “accidentally” triggers this effect, and order the ad placed on digital billboards in Silicon Valley through a fake company located somewhere far far away.
- userbinator 6y ago
  There have already been (audio) ads that trigger speech-recognising machines like Alexa, and in that case I would very much say it is free speech.
  - MaxBarraclough 6y ago
    Like the famous 2017 Burger King ad that deliberately hijacked nearby Google Home smart speakers. [0] There's an argument to be made that it should be covered by cyber-attack laws. You aren't allowed to deliberately take control of someone else's computer without their permission. Consider a hypothetical example: an author without use of their hands, who uses text-to-speech to write books and to control their computer. Imagine if they put you on speakerphone and you read out a sequence of commands to delete their work. That certainly wouldn't be covered under freedom of expression. There's an obvious difference in degree, as spamming someone by hijacking their smart speaker isn't all that costly, but it's similar in category. [0] https://en.wikipedia.org/wiki/Burger_King_advertising#2000s%E2%80%932010s https://en.wikipedia.org/wiki/Burger_King_advertising#2000s%...
    - dorgo 6y ago
      >There's an argument to be made that it should be covered by cyber-attack laws. Yes, but smart speakers should not rely on laws to prevent successfull attacks.
quickthrower2 6y ago
Low chance of any justice.
gridlockd 6y ago
Your run-of-the-mill criminal has some sort of profit motive and this seems like a very roundabout way to attempt extortion. A terrorist on the other hand would probably be dissatisfied with the not-quite-catastophic outcome. If you want to cause real havoc, there are far more valuable targets to hack into.
- josefx 6y ago
  We have idiots that call swat teams on twitch streamers. The profit is for the lulz.
  - gridlockd 6y ago
    Yes, any idiot can call 911 and have SWAT team dispatched, but that is a very low effort "attack". I'm not saying that this could never possibly happen, but it's not much of a real threat exactly because effort vs reward is unfavorable.
handol 6y ago
What about the criminal who mounted the screen next to the road and hooked it up to the internet?

accurrent 6y ago

To be fair I think autonomous vehicles should have a requirement that they come with a RADAR or LIDAR as a backup sensor (even if its a low resolution one). Using ML on images for everything is not going to solve your safety problems cause theres no way to gaurantee the system will not crashed when presented with a fake input. Its a darn sight harder to pretend to be farther away on a radar or lidar though. At the very least the autonomous vehicle should be able to gaurantee you don't crash into static obstacles.

hellotomyrars 6y ago
As far as I know most people working in the space still think that way, with the very notable exception of Tesla.

Balgair 6y ago

Why would that be a crime? The driver is ultimately responsible, not the car.

mattbee 6y ago
If you know that displaying unofficial highway signs will cause "self-driving" cars to swerve off the road, it's not much different an act from dropping breeze blocks off a motorway bridge. I've no idea whether that'd be a highway crime, a computer misuse crime, or just attempted murder - that'd be a new one for lawyers in every country to argue.

jacobsenscott 6y ago

There are already billboards with pictures of stop signs, stop lights, etc on them. It doesn't take any criminal intent. Things that were once just billboards are suddenly hazards.

im3w1l 6y ago
Seems reasonable enough to ban that. Might confuse a tired human too.
- Ekaros 6y ago
  I think banning Teslas as unsafe is even more reasonable conclusion. Those cars can't be trusted so they should not be allowed on public roads. Problem solved. Same goes for any other car with this issue.
  - 542354234235 6y ago
    Or human drivers for that matter. I heard that they have a very high failure rate and cause over 6,500,000 collisions a year in the US alone. Take them back to the drawing board and work out all the bugs until they can put a safe species of driver on the road.
- jfim 6y ago
  Or maybe remind them to stop for school buses: https://www.wkyt.com/content/news/New-billboards-to-remind-drivers-to-stop-for-school-buses-436832083.html https://www.wkyt.com/content/news/New-billboards-to-remind-d...

onion2k 6y ago

Maybe Lucid will an advert that deliberately makes Teslas stop in front of it.