7 ms·

I think the concept of the line of death, from browser security UI design, is relevant here. The space outside of the highway right of way is completely uncontr

I think the concept of the line of death, from browser security UI design, is relevant here. The space outside of the highway right of way is completely uncontrolled and the Tesla should know better than to look at a billboard hundreds of feet up in the air for regulatory signs. The MUTCD has rules about where signs are to be placed--look only in those places. [1] https://textslashplain.com/2017/01/14/the-line-of-death/ https://textslashplain.com/2017/01/14/the-line-of-death/

shajznnckfke 6y ago
The problem is that they don’t have lidar and therefore can’t reliably measure depth, right? A billboard far away high up in the air may appear in the same direction as a closer road sign if you don’t have depth perception. You should be able to infer it from a camera with the right model but those methods are more fallible. Similar root cause to that time the guy got beheaded by a white truck perpendicular to the highway that looked like the sky.
- kirillzubovsky 6y ago
  I believe they are making depth perception using computer vision. The idea is that if you can take an image and predict the distances to the objects in it, you can then create an image that is a transpose of the original based on your predictions, but at a different distance. In case of Tesla moving forward, that would be a split second later, closer to the original image. As the car moves forward, you actually take that second image and compare the virtual to the original. At this point you probably get it wrong, but you adjust your predictions and recalculate until your virtual transposed image looks exactly like the real image, at which point you've got the distances pretty darn close. I remember watching a talk about their computer vision, and it was pretty impressive. The challenge is that despite knowing where the billboards are, they still look that way because sometimes, in obscure situations, in little old towns with fuzzy rules, road signs actually do appear in weird places, and that's where the car often trips up because without sufficient data it just can't build a prediction model to know when to respond, and when to suppress the new input.
  - kavalg 6y ago
    It is called depth from motion. I think this is what the Google guys used to create the 3D buildings view back then.
    - viggity 6y ago
      and depth from motion is really just stereoscopy using multiple photos from the same camera instead of two photos from two different cameras. I have a side project where I manage to get a helluva lot of depth info using a single photo from a single camera (monocular photogrammetry).
- Acinyx 6y ago
  They put this "billboard" right at the height and location of a normal stop sign, location is not going to help for this.
raxxorrax 6y ago
If you don't evaluate everything your camera sees, you basically just decrease its FOV. Sure, that might net you less errors for objects that are most relevant most of the time. But it is the unexpected we need to train computer vision for to compete with humans. We can have rules for things like billboards, sure, but those will never be completely implemented everywhere. The software just needs to be improved to identify something as billboard.
godelski 6y ago
Someone else linked this[0] article that has a video of the attack. I think seeing the video will paint a different picture than what you have in your head. The "billboard" here is in a location that a stop sign would be, not your average highway billboard. One would hope Tesla's system has some depth awareness. Honestly I'd be impressed how they could do so much if it didn't. [0] https://www.wired.com/story/tesla-model-x-autopilot-phantom-images/ https://www.wired.com/story/tesla-model-x-autopilot-phantom-...
- AdamHede 6y ago
  This is pretty important. The way the guys does this is very close to how you could also fool a human by hanging up real stop signs in the wrong places, and "stupidity" people would start to respect them. The only difference here is that a mere 500 ms flash elicites a response, where a human wouldn't notice. on balance, a 500 ms response time does seem like a feature more than a bug compared to a human.
  - bipson 6y ago
    Well, which one is it now: Are self-driving computers superior to humans and make less mistakes, or is it OK for a computer to be tricked by something some humans would also not detect? Btw. I think I can tell if there is a real stop-sign or something on a billboard after some 'huh'-moment and without slamming on the brakes. And humans still go circles around AI regarding plausibility-checking very out-of-the-ordinary events.
    - 542354234235 6y ago
      It isn’t either/or. One system can be overall superior another and that system can have different vulnerabilities and weaknesses. Cars are generally considered better than horses for transport, but cars also cannot use grass for fuel.
    - mjburgess 6y ago
      It's neither. Self-driving cars aren't superior. There is 1 death in 100 million miles driven on roads; self-driving cars are currently operating around 1 death per million. Personally, I do not expect self-driving cars this century to get anywhere near human safety levels (assuming cities/etc. remain as-is).

shajznnckfke 6y ago

The problem is that they don’t have lidar and therefore can’t reliably measure depth, right? A billboard far away high up in the air may appear in the same direction as a closer road sign if you don’t have depth perception. You should be able to infer it from a camera with the right model but those methods are more fallible. Similar root cause to that time the guy got beheaded by a white truck perpendicular to the highway that looked like the sky.

kirillzubovsky 6y ago
I believe they are making depth perception using computer vision. The idea is that if you can take an image and predict the distances to the objects in it, you can then create an image that is a transpose of the original based on your predictions, but at a different distance. In case of Tesla moving forward, that would be a split second later, closer to the original image. As the car moves forward, you actually take that second image and compare the virtual to the original. At this point you probably get it wrong, but you adjust your predictions and recalculate until your virtual transposed image looks exactly like the real image, at which point you've got the distances pretty darn close. I remember watching a talk about their computer vision, and it was pretty impressive. The challenge is that despite knowing where the billboards are, they still look that way because sometimes, in obscure situations, in little old towns with fuzzy rules, road signs actually do appear in weird places, and that's where the car often trips up because without sufficient data it just can't build a prediction model to know when to respond, and when to suppress the new input.
- kavalg 6y ago
  It is called depth from motion. I think this is what the Google guys used to create the 3D buildings view back then.
  - viggity 6y ago
    and depth from motion is really just stereoscopy using multiple photos from the same camera instead of two photos from two different cameras. I have a side project where I manage to get a helluva lot of depth info using a single photo from a single camera (monocular photogrammetry).
Acinyx 6y ago
They put this "billboard" right at the height and location of a normal stop sign, location is not going to help for this.

raxxorrax 6y ago

If you don't evaluate everything your camera sees, you basically just decrease its FOV. Sure, that might net you less errors for objects that are most relevant most of the time. But it is the unexpected we need to train computer vision for to compete with humans. We can have rules for things like billboards, sure, but those will never be completely implemented everywhere. The software just needs to be improved to identify something as billboard.

godelski 6y ago

Someone else linked this[0] article that has a video of the attack. I think seeing the video will paint a different picture than what you have in your head. The "billboard" here is in a location that a stop sign would be, not your average highway billboard. One would hope Tesla's system has some depth awareness. Honestly I'd be impressed how they could do so much if it didn't. [0] https://www.wired.com/story/tesla-model-x-autopilot-phantom-images/ https://www.wired.com/story/tesla-model-x-autopilot-phantom-...

AdamHede 6y ago
This is pretty important. The way the guys does this is very close to how you could also fool a human by hanging up real stop signs in the wrong places, and "stupidity" people would start to respect them. The only difference here is that a mere 500 ms flash elicites a response, where a human wouldn't notice. on balance, a 500 ms response time does seem like a feature more than a bug compared to a human.
- bipson 6y ago
  Well, which one is it now: Are self-driving computers superior to humans and make less mistakes, or is it OK for a computer to be tricked by something some humans would also not detect? Btw. I think I can tell if there is a real stop-sign or something on a billboard after some 'huh'-moment and without slamming on the brakes. And humans still go circles around AI regarding plausibility-checking very out-of-the-ordinary events.
  - 542354234235 6y ago
    It isn’t either/or. One system can be overall superior another and that system can have different vulnerabilities and weaknesses. Cars are generally considered better than horses for transport, but cars also cannot use grass for fuel.
  - mjburgess 6y ago
    It's neither. Self-driving cars aren't superior. There is 1 death in 100 million miles driven on roads; self-driving cars are currently operating around 1 death per million. Personally, I do not expect self-driving cars this century to get anywhere near human safety levels (assuming cities/etc. remain as-is).