5 ms·

> It seems a little weird to focus on this. I'm not sure I'd say that this is being "focused" on. It's an interesting bit of info, the other players in the sp

by Lazare 9y ago

> It seems a little weird to focus on this. I'm not sure I'd say that this is being "focused" on. It's an interesting bit of info, the other players in the space (Intel, Volvo, Aptiv) have an interest in getting their spin out there, readers are interested, so we get this article and everyone is happy. It's not a super critical key to the story, but it does help explain what happened in some small way. > having a second system running that can't coordinate with the self-driving system would likely make the combined system less safe. That seems like a bit of a leap to me. The Volvo system is not super-advanced; the relevant bit here would probably be their "Collision Warning with Auto Brake" feature. Multiple independent systems with the ability to sound an alarm and slam on the brakes if they detect an imminent pedestrian collision...seem like they probably would be safer overall? They'd certainly be safer for the pedestrians, and we're currently at 1 pedestrian fatality and 0 fatalities for "people in cars killed due to overzealous/conflicting automated safety systems". I rather suspect Uber disabled it because they wanted better data, not because they thought it was safer for other people on the road. That's certainly a valid choice, but one which might have been wiser if their tech was a lot more reliable.

majewsky 9y ago
> we're currently at 1 pedestrian fatality and 0 fatalities for "people in cars killed due to overzealous/conflicting automated safety systems". Not trying to take sides here, but the last part strikes me as "citation needed".
- Lazare 9y ago
  Sorry, I guess I left out the implicit "as far as I know". Do you know of anyone who's died that way?
  - dotsh 9y ago
    Yes: https://www.theguardian.com/technology/2016/jun/30/tesla-autopilot-death-self-driving-car-elon-musk https://www.theguardian.com/technology/2016/jun/30/tesla-aut...
    - majewsky 9y ago
      That's neither a "overzealous/conflicting" system nor a "safety system", so if anything, this would bring the score to 2:0 rather than 1:1.
    - Lazare 9y ago
      I was already aware of that one, but sure, that's another example of the opposite of what I asked for. Rather than someone killed by conflicting or over-cautious safety systems, that's an example of someone killed by a single ineffective safety system, just like the pedestrian in Arizona. This further reinforces my point, I think. :)
jschwartzi 9y ago
Disabling a backup system like that strikes me as very negligent from an engineering perspective. They had reason to expect their system to fail because it's a prototype, but then they go and disable another system that is actively marketed and that could have prevented this death. I'm so glad they're not testing their cars in my state. This makes me wonder if they even do basic safety engineering like FMEA or if they just wing it.
- Already__Taken 9y ago
  But it was never built as a backup system. It's a whole feature that was designed from the ground up assuming it's the only thing on the car doing anything like the job it does. There's 2 chefs.
  - PinguTS 9y ago
    Actually, those systems are designed as backup. They are designed as backup, when the driver does not react. That is the reason why they warn first and initiate an emergency brake only, and only if an accident becomes unavoidable. So that would, if the automated tech from Uber would have detected in time, it would have reacted like a human driver and the pre-installed system would may have not noticed it. But in case, it would have reacted as a backup.
    - kylegordon 9y ago
      On the two occasions my Volvo has decided to take over (both admittedly due to overhanging bushes in a parking space), it has taken over control immediately and entirely. There is no warning.
      - tzakrajs 9y ago
        No warning necessary because self driving cars are completely capable of being programmed to deal with varying ability to control the car. It can still have an intention whether or not the car is allowing it to proceed. It needs to make a best effort, allowing failure when a backup system takes control.
w0m 9y ago
> Multiple independent systems with the ability to sound an alarm and slam on the brakes if they detect an imminent pedestrian collision...seem like they probably would be safer overall? While yes - it would also force Uber to permanently partner with Volvo on technology they can't control. 'Uhuh, my Ubers a Chevy not a Ford - it could kill me because the Chevies don't built-in X is worse!'. You do not want an arbitrary third party system controlling the car while the Uber system is in unknown ways; if the systems conflicted things could go out of control and be more dangerous.
- mannykannot 9y ago
  > it would also force Uber to permanently partner with Volvo on technology they can't control. That sounds like a win-win to me. >You do not want an arbitrary third party system controlling the car while the Uber system is in unknown ways; if the systems conflicted things could go out of control and be more dangerous. I find it interesting that, of these two systems, it is Volvo's widely-deployed system that is described as 'arbitrary'. If this argument had any merit, it would be unsafe for this system to be on the road, 'conflicting' with the car's human driver. On the contrary, the statistics clearly show that these systems reduce the frequency and severity of accidents.
  - w0m 9y ago
    I know this sounds bad, but the arbitrary is that the Ubers aren't designed to be as safe as possible currently - they're designed to be testbeds of the new fully autonomous technology uber is developing. Maybe it makes sense to rely on arbitrary vehicle specific fallbacks, but long-term it doesn't as I'm sure They don't want to be dependent on a specific model. When you request a ride, do you want to care about whether your getting a Honda or a Volvo?

by Lazare 9y ago

majewsky 9y ago

> we're currently at 1 pedestrian fatality and 0 fatalities for "people in cars killed due to overzealous/conflicting automated safety systems". Not trying to take sides here, but the last part strikes me as "citation needed".

Lazare 9y ago
Sorry, I guess I left out the implicit "as far as I know". Do you know of anyone who's died that way?
- dotsh 9y ago
  Yes: https://www.theguardian.com/technology/2016/jun/30/tesla-autopilot-death-self-driving-car-elon-musk https://www.theguardian.com/technology/2016/jun/30/tesla-aut...
  - majewsky 9y ago
    That's neither a "overzealous/conflicting" system nor a "safety system", so if anything, this would bring the score to 2:0 rather than 1:1.
  - Lazare 9y ago
    I was already aware of that one, but sure, that's another example of the opposite of what I asked for. Rather than someone killed by conflicting or over-cautious safety systems, that's an example of someone killed by a single ineffective safety system, just like the pedestrian in Arizona. This further reinforces my point, I think. :)

jschwartzi 9y ago

Disabling a backup system like that strikes me as very negligent from an engineering perspective. They had reason to expect their system to fail because it's a prototype, but then they go and disable another system that is actively marketed and that could have prevented this death. I'm so glad they're not testing their cars in my state. This makes me wonder if they even do basic safety engineering like FMEA or if they just wing it.

Already__Taken 9y ago
But it was never built as a backup system. It's a whole feature that was designed from the ground up assuming it's the only thing on the car doing anything like the job it does. There's 2 chefs.
- PinguTS 9y ago
  Actually, those systems are designed as backup. They are designed as backup, when the driver does not react. That is the reason why they warn first and initiate an emergency brake only, and only if an accident becomes unavoidable. So that would, if the automated tech from Uber would have detected in time, it would have reacted like a human driver and the pre-installed system would may have not noticed it. But in case, it would have reacted as a backup.
  - kylegordon 9y ago
    On the two occasions my Volvo has decided to take over (both admittedly due to overhanging bushes in a parking space), it has taken over control immediately and entirely. There is no warning.
    - tzakrajs 9y ago
      No warning necessary because self driving cars are completely capable of being programmed to deal with varying ability to control the car. It can still have an intention whether or not the car is allowing it to proceed. It needs to make a best effort, allowing failure when a backup system takes control.

w0m 9y ago

> Multiple independent systems with the ability to sound an alarm and slam on the brakes if they detect an imminent pedestrian collision...seem like they probably would be safer overall? While yes - it would also force Uber to permanently partner with Volvo on technology they can't control. 'Uhuh, my Ubers a Chevy not a Ford - it could kill me because the Chevies don't built-in X is worse!'. You do not want an arbitrary third party system controlling the car while the Uber system is in unknown ways; if the systems conflicted things could go out of control and be more dangerous.

mannykannot 9y ago
> it would also force Uber to permanently partner with Volvo on technology they can't control. That sounds like a win-win to me. >You do not want an arbitrary third party system controlling the car while the Uber system is in unknown ways; if the systems conflicted things could go out of control and be more dangerous. I find it interesting that, of these two systems, it is Volvo's widely-deployed system that is described as 'arbitrary'. If this argument had any merit, it would be unsafe for this system to be on the road, 'conflicting' with the car's human driver. On the contrary, the statistics clearly show that these systems reduce the frequency and severity of accidents.
- w0m 9y ago
  I know this sounds bad, but the arbitrary is that the Ubers aren't designed to be as safe as possible currently - they're designed to be testbeds of the new fully autonomous technology uber is developing. Maybe it makes sense to rely on arbitrary vehicle specific fallbacks, but long-term it doesn't as I'm sure They don't want to be dependent on a specific model. When you request a ride, do you want to care about whether your getting a Honda or a Volvo?