5 ms·

It boggles my mind that any self driving system would knowingly drive into any object, no matter what it is, as large as a police car. I'd be willing to bet th

It boggles my mind that any self driving system would knowingly drive into any object, no matter what it is, as large as a police car. I'd be willing to bet that Elon's insistence on only using cameras is combining with the flashing lights to effectively make objects that appear in some frames of video, and not in others, as the camera recovers from the flash... and this makes it appear as noise, and below a certain threshold, ignored. Or, it could be the reflective decals... who knows.

sheepscreek 2y ago
AFAIK the latest betas have switched to a cutting edge “transformer” model (or models?). I wonder if it’s prone to hallucinations like its LLM cousins. On a different note, I like how Mercedes changes the color of external lights when in self-driving mode to make others aware. A bit like outsourcing the responsibility externally - but maybe this is the way forward until the tech is reliable/better than people.
- t0mas88 2y ago
  I like the Mercedes idea, but haven't seen it in practice yet. It would be useful to have a universal outside light that informs drivers behind you that the reason you're keeping a safe distance ahead is that the car is on adaptive cruise control. Some people feel like 2 seconds distance is a reason to try crazy stunts to get around you. And then once the road is clear drive slower in front of you than the speed you had set...
- Rinzler89 2y ago
  >I wonder if it’s prone to hallucinations like its LLM cousins. When was their vision system not glitching? A year ago it was seeing the moon on the sky as a perpetual yellow traffic light.
  - yreg 2y ago
    That's not hallucination, that's image recognition issue.
    - Rinzler89 2y ago
      How do you define the border between hallucination vs recognition issue?
      - temporarely 2y ago
        Isolating an object is baseline recognition. Any sort of semantic association is baseline interpretation. So that border is somewhere between "light source" and "traffic light".
      - yreg 2y ago
        I thought hallucination is when the model imagines something that isn't there at all. Not a classification issue where it misinterprets an object in the source data. Sometimes it might be difficult to tell, but in the case of moon -> traffic light it seems clear. Maybe I'm wrong about the terminology.
- dzhiurgis 2y ago
  transformer and LLM aren't cousins. it's like saying transistor and cpu is a cousin...
radu_floricica 2y ago
I mean, it's to be expected. Most driving infrastructure is designed for human eyes and brains, including flashing lights on police cars. If you redesign the driving system from scratch to work on silicon and not brain, you pretty much expect its failures to not look like the kind of failures we're used to. Even if it's objectively better, with a lower accident rate, lower fatality rate and so on, it will occasionally do something as stupid looking as "drive into a police car with flashing lights on". Which is why suitability decisions should not be made based on anecdotes.
- JumpCrisscross 2y ago
  > redesign the driving system from scratch to work on silicon Nobody proposed this. My Subaru can lane keep even in whiteout conditions (if there is a car in front of me) due to radar. I can’t do that. Tesla’s can’t do that. > it will occasionally do something as stupid looking as "drive into a police car with flashing lights on" Two sensors operating on different wavelengths can add more information than a single doubling of the first sensor.
  - hughesjj 2y ago
    If I could have lidar eyes in addition to my normal ones you bet I'd go for it in the instance. Same with more variety of cones or thermal or any of the other cool sensory systems some other animals have. Tesla took out the non-visible detectors for cost savings alone. I don't get anyone who every believed otherwise.
  - nradov 2y ago
    Huh? The Subaru EyeSight system uses only cameras. There is no radar. https://www.subaru.com/eyesight.html https://www.subaru.com/eyesight.html IME it works pretty well but cuts out in heavy precipitation because the cameras don't get a clear image.
    - JumpCrisscross 2y ago
      EyeSight is not the only lane-keeping component.
deleted 2y ago
[deleted]
toast0 2y ago
First off, Tesla 'self driving' isn't a self driving system. Read their letters to the California DMV and ignore their marketting, including their choice of names. Second, radar wouldn't be better here. Afaik, all of the radar assisted cruise control / emergency braking systems ignore stationary objects when travelling above a certain speed. Using radar, it's difficult to determine the height or shape of a potential obstruction, so a car on the road looks a lot like a metal sign above the roadway. Lidar is much better, but cost prohibitive to put onto vehicles for driver assistance. A camera system could possibly detect stationary objects of concern, but apparently doesn't.
- gamblor956 2y ago
  Afaik, all of the radar assisted cruise control / emergency braking systems ignore stationary objects when travelling above a certain speed. This is false. Radar assisted cruise control definitely do not ignore stationary objects above a certain speed; indeed, detecting stationary objects is a fundamental part of how emergency braking systems work... Other automakers are simply better at determining what objects are in the path of travel than Tesla, because they actually test their systems before they release them to the public.
  - toast0 2y ago
    This article [1] is several years old, but I believe it is still an accurate summary: > Sam Abuelsamid, an industry analyst at Navigant and former automotive engineer, tells Ars that it's "pretty much universal" that "vehicles are programmed to ignore stationary objects at higher speeds." Chevrolet says [2] > At speeds between 5 and 50 mph, Automatic Emergency Braking (AEB) can help you avoid or reduce the severity of a collision† with a detected vehicle you’re following using camera technology. It can automatically provide hard emergency braking or enhance the driver’s hard braking. They don't specifically say it won't detect stationary objects on this page, but a vehicle you're following will have been in motion, or you're not following it. Consumer reports describes several types of AEB [3], none of which mention stationary objects. If you've got a reference that suggests that driver assistance features stop for stationary objects outside of parking speeds, please provide a link. [1] https://arstechnica.com/cars/2018/06/why-emergency-braking-systems-sometimes-hit-parked-cars-and-lane-dividers/ https://arstechnica.com/cars/2018/06/why-emergency-braking-s... [2] https://www.chevrolet.com/support/vehicle/driving-safety/brakes/enhanced-automatic-emergency-braking https://www.chevrolet.com/support/vehicle/driving-safety/bra... [3] https://www.consumerreports.org/cars/car-safety/automatic-emergency-braking-guide-a1780056935/ https://www.consumerreports.org/cars/car-safety/automatic-em...
aaron695 2y ago
[dead]
diggernet 2y ago
The problem is that Tesla uses an ML classifier to identify objects it sees. Once it has identified them, it can estimate how big it is, and thus how far away it is. But if the classifier doesn't identify the object, it might as well not exist. Contrast this with Subaru, who uses binocular view through two widely spaced cameras to build a depth map of the view in front of it. It doesn't know or care what is there, only that something is there it shouldn't hit. And so the emergency breaking will trigger for things like trains and stationary emergency vehicles that Tesla seems oblivious to. Sometimes simpler really is better.
- eurleif 2y ago
  This is false. Tesla has an occupancy network.
  - diggernet 2y ago
    Please pardon my failure to remember the correct term. But fundamentally, it constructs a 3d model based on what it thinks each pixel represents. And sometimes gets it wrong, leading to it having a pattern of driving into stationary objects.
    - eurleif 2y ago
      It's not a matter of terminology. Tesla's occupancy network "doesn't know or care what is there, only that something is there it shouldn't hit". As the name suggests, it's solely concerned with what space is occupied and what space is unoccupied, not with identifying objects. As such, the distinction you tried to draw between Subaru and Tesla is false, no matter what terms you use.
jojobas 2y ago
It learned from reddit that cops aren't people.
SmartJerry 2y ago
Right, but we are relying on what the 'driver said.' Anyone who has been involved in an accident knows the other driver will lie through their teeth to avoid responsibility.

sheepscreek 2y ago

AFAIK the latest betas have switched to a cutting edge “transformer” model (or models?). I wonder if it’s prone to hallucinations like its LLM cousins. On a different note, I like how Mercedes changes the color of external lights when in self-driving mode to make others aware. A bit like outsourcing the responsibility externally - but maybe this is the way forward until the tech is reliable/better than people.

t0mas88 2y ago
I like the Mercedes idea, but haven't seen it in practice yet. It would be useful to have a universal outside light that informs drivers behind you that the reason you're keeping a safe distance ahead is that the car is on adaptive cruise control. Some people feel like 2 seconds distance is a reason to try crazy stunts to get around you. And then once the road is clear drive slower in front of you than the speed you had set...
Rinzler89 2y ago
>I wonder if it’s prone to hallucinations like its LLM cousins. When was their vision system not glitching? A year ago it was seeing the moon on the sky as a perpetual yellow traffic light.
- yreg 2y ago
  That's not hallucination, that's image recognition issue.
  - Rinzler89 2y ago
    How do you define the border between hallucination vs recognition issue?
    - temporarely 2y ago
      Isolating an object is baseline recognition. Any sort of semantic association is baseline interpretation. So that border is somewhere between "light source" and "traffic light".
    - yreg 2y ago
      I thought hallucination is when the model imagines something that isn't there at all. Not a classification issue where it misinterprets an object in the source data. Sometimes it might be difficult to tell, but in the case of moon -> traffic light it seems clear. Maybe I'm wrong about the terminology.
dzhiurgis 2y ago
transformer and LLM aren't cousins. it's like saying transistor and cpu is a cousin...

radu_floricica 2y ago

I mean, it's to be expected. Most driving infrastructure is designed for human eyes and brains, including flashing lights on police cars. If you redesign the driving system from scratch to work on silicon and not brain, you pretty much expect its failures to not look like the kind of failures we're used to. Even if it's objectively better, with a lower accident rate, lower fatality rate and so on, it will occasionally do something as stupid looking as "drive into a police car with flashing lights on". Which is why suitability decisions should not be made based on anecdotes.

JumpCrisscross 2y ago
> redesign the driving system from scratch to work on silicon Nobody proposed this. My Subaru can lane keep even in whiteout conditions (if there is a car in front of me) due to radar. I can’t do that. Tesla’s can’t do that. > it will occasionally do something as stupid looking as "drive into a police car with flashing lights on" Two sensors operating on different wavelengths can add more information than a single doubling of the first sensor.
- hughesjj 2y ago
  If I could have lidar eyes in addition to my normal ones you bet I'd go for it in the instance. Same with more variety of cones or thermal or any of the other cool sensory systems some other animals have. Tesla took out the non-visible detectors for cost savings alone. I don't get anyone who every believed otherwise.
- nradov 2y ago
  Huh? The Subaru EyeSight system uses only cameras. There is no radar. https://www.subaru.com/eyesight.html https://www.subaru.com/eyesight.html IME it works pretty well but cuts out in heavy precipitation because the cameras don't get a clear image.
  - JumpCrisscross 2y ago
    EyeSight is not the only lane-keeping component.

deleted 2y ago

[deleted]

toast0 2y ago

First off, Tesla 'self driving' isn't a self driving system. Read their letters to the California DMV and ignore their marketting, including their choice of names. Second, radar wouldn't be better here. Afaik, all of the radar assisted cruise control / emergency braking systems ignore stationary objects when travelling above a certain speed. Using radar, it's difficult to determine the height or shape of a potential obstruction, so a car on the road looks a lot like a metal sign above the roadway. Lidar is much better, but cost prohibitive to put onto vehicles for driver assistance. A camera system could possibly detect stationary objects of concern, but apparently doesn't.

gamblor956 2y ago
Afaik, all of the radar assisted cruise control / emergency braking systems ignore stationary objects when travelling above a certain speed. This is false. Radar assisted cruise control definitely do not ignore stationary objects above a certain speed; indeed, detecting stationary objects is a fundamental part of how emergency braking systems work... Other automakers are simply better at determining what objects are in the path of travel than Tesla, because they actually test their systems before they release them to the public.
- toast0 2y ago
  This article [1] is several years old, but I believe it is still an accurate summary: > Sam Abuelsamid, an industry analyst at Navigant and former automotive engineer, tells Ars that it's "pretty much universal" that "vehicles are programmed to ignore stationary objects at higher speeds." Chevrolet says [2] > At speeds between 5 and 50 mph, Automatic Emergency Braking (AEB) can help you avoid or reduce the severity of a collision† with a detected vehicle you’re following using camera technology. It can automatically provide hard emergency braking or enhance the driver’s hard braking. They don't specifically say it won't detect stationary objects on this page, but a vehicle you're following will have been in motion, or you're not following it. Consumer reports describes several types of AEB [3], none of which mention stationary objects. If you've got a reference that suggests that driver assistance features stop for stationary objects outside of parking speeds, please provide a link. [1] https://arstechnica.com/cars/2018/06/why-emergency-braking-systems-sometimes-hit-parked-cars-and-lane-dividers/ https://arstechnica.com/cars/2018/06/why-emergency-braking-s... [2] https://www.chevrolet.com/support/vehicle/driving-safety/brakes/enhanced-automatic-emergency-braking https://www.chevrolet.com/support/vehicle/driving-safety/bra... [3] https://www.consumerreports.org/cars/car-safety/automatic-emergency-braking-guide-a1780056935/ https://www.consumerreports.org/cars/car-safety/automatic-em...

aaron695 2y ago

[dead]

diggernet 2y ago

The problem is that Tesla uses an ML classifier to identify objects it sees. Once it has identified them, it can estimate how big it is, and thus how far away it is. But if the classifier doesn't identify the object, it might as well not exist. Contrast this with Subaru, who uses binocular view through two widely spaced cameras to build a depth map of the view in front of it. It doesn't know or care what is there, only that something is there it shouldn't hit. And so the emergency breaking will trigger for things like trains and stationary emergency vehicles that Tesla seems oblivious to. Sometimes simpler really is better.

eurleif 2y ago
This is false. Tesla has an occupancy network.
- diggernet 2y ago
  Please pardon my failure to remember the correct term. But fundamentally, it constructs a 3d model based on what it thinks each pixel represents. And sometimes gets it wrong, leading to it having a pattern of driving into stationary objects.
  - eurleif 2y ago
    It's not a matter of terminology. Tesla's occupancy network "doesn't know or care what is there, only that something is there it shouldn't hit". As the name suggests, it's solely concerned with what space is occupied and what space is unoccupied, not with identifying objects. As such, the distinction you tried to draw between Subaru and Tesla is false, no matter what terms you use.

jojobas 2y ago

It learned from reddit that cops aren't people.

SmartJerry 2y ago

Right, but we are relying on what the 'driver said.' Anyone who has been involved in an accident knows the other driver will lie through their teeth to avoid responsibility.