5 ms·

But it was never built as a backup system. It's a whole feature that was designed from the ground up assuming it's the only thing on the car doing anything like

But it was never built as a backup system. It's a whole feature that was designed from the ground up assuming it's the only thing on the car doing anything like the job it does. There's 2 chefs.

PinguTS 9y ago
Actually, those systems are designed as backup. They are designed as backup, when the driver does not react. That is the reason why they warn first and initiate an emergency brake only, and only if an accident becomes unavoidable. So that would, if the automated tech from Uber would have detected in time, it would have reacted like a human driver and the pre-installed system would may have not noticed it. But in case, it would have reacted as a backup.
- kylegordon 9y ago
  On the two occasions my Volvo has decided to take over (both admittedly due to overhanging bushes in a parking space), it has taken over control immediately and entirely. There is no warning.
  - tzakrajs 9y ago
    No warning necessary because self driving cars are completely capable of being programmed to deal with varying ability to control the car. It can still have an intention whether or not the car is allowing it to proceed. It needs to make a best effort, allowing failure when a backup system takes control.
Lazare 9y ago
> But it was never built as a backup system. The collision avoidance system? The fail-safe that's meant to ensure the car doesn't hit something when the human driver screws up? The one that works with an entirely separate autonomous intelligence (aka, the human) controlling the car? The one explicitly designed to trigger emergency brakeing even if the primary controller is trying to do something else? The kind of system installed on many other AI controlled cars to avoid this exact type of disaster? That backup system? Look, it's entirely possible that for some reason this specific safety system conflicted with the current version of Uber's system, but we have zero data to suggest that, and if so, it's something specific to those two systems in that particular vehicle (and suggests a severe issue with Uber's system to boot). There's no general issue here. > There's 2 chefs. Yes, and sometimes one of them takes over from the other one. That's how this works.
- BoorishBears 9y ago
  Have you actually driven with one of these systems? There are plenty of intentional maneuvers required to integrate with real-world (ie. non perfect) traffic that trigger these systems. They’re designed to take hints from human input (so for example, LKA responds less strongly to forceful unexpected inputs than a lack of expected input). There is 0 chance the system wouldn’t interfere with a self driving. Not to mention many of those safety systems include lockouts on the CAN bus which make removing their safety module a first step to even add your own inputs. I know that’s how it works on my Volt with a Mobileye sensor
  - tzakrajs 9y ago
    The self driving should be capable of mitigating this risk by adapting to systems like these in general.
    - fixermark 9y ago
      Problem is, that greatly complicates the problem-solviing process for SD vehicles---you're now trying to build atop a model of the world provided by the vendor, with its own faults and quirks (and you probably need an agreement with the vendor to see their code / dissect their model, and now your release schedule depends on their release schedule, etc.). It also makes Uber dependent upon Volvo's buy-in on the project, and that arrangement is fraught in a world where we don't yet know if SD cars are going to damage the business of traditional auto manufacturers 50 years out---Uber wants to be in a place where if a vendor relationship sours, they can pack up and change chassis without a complete overhaul of the SD logic. If your SD system is supposed to be solving 100% of the same problem anyway, it's best to gut the software you don't control and work with the hardware as directly as you can. The notion of building atop a lower-level abstraction makes for great academia but doesn't work with industrial realities.
      - tzakrajs 9y ago
        Thanks for the explanation and I agree that it won't be as easy to build SD for auto braking systems that change from chassis to chassis, but this is what I think people want. People want to have fail safes from different vendors.
kwhitefoot 9y ago
Automatic emergency braking is quite definitely a backup system. If it ever trips in my car it is because the primary system, that is me, has failed to brake in time.

But it was never built as a backup system. It's a whole feature that was designed from the ground up assuming it's the only thing on the car doing anything like the job it does. There's 2 chefs.

PinguTS 9y ago

Actually, those systems are designed as backup. They are designed as backup, when the driver does not react. That is the reason why they warn first and initiate an emergency brake only, and only if an accident becomes unavoidable. So that would, if the automated tech from Uber would have detected in time, it would have reacted like a human driver and the pre-installed system would may have not noticed it. But in case, it would have reacted as a backup.

kylegordon 9y ago
On the two occasions my Volvo has decided to take over (both admittedly due to overhanging bushes in a parking space), it has taken over control immediately and entirely. There is no warning.
- tzakrajs 9y ago
  No warning necessary because self driving cars are completely capable of being programmed to deal with varying ability to control the car. It can still have an intention whether or not the car is allowing it to proceed. It needs to make a best effort, allowing failure when a backup system takes control.

Lazare 9y ago

> But it was never built as a backup system. The collision avoidance system? The fail-safe that's meant to ensure the car doesn't hit something when the human driver screws up? The one that works with an entirely separate autonomous intelligence (aka, the human) controlling the car? The one explicitly designed to trigger emergency brakeing even if the primary controller is trying to do something else? The kind of system installed on many other AI controlled cars to avoid this exact type of disaster? That backup system? Look, it's entirely possible that for some reason this specific safety system conflicted with the current version of Uber's system, but we have zero data to suggest that, and if so, it's something specific to those two systems in that particular vehicle (and suggests a severe issue with Uber's system to boot). There's no general issue here. > There's 2 chefs. Yes, and sometimes one of them takes over from the other one. That's how this works.

BoorishBears 9y ago
Have you actually driven with one of these systems? There are plenty of intentional maneuvers required to integrate with real-world (ie. non perfect) traffic that trigger these systems. They’re designed to take hints from human input (so for example, LKA responds less strongly to forceful unexpected inputs than a lack of expected input). There is 0 chance the system wouldn’t interfere with a self driving. Not to mention many of those safety systems include lockouts on the CAN bus which make removing their safety module a first step to even add your own inputs. I know that’s how it works on my Volt with a Mobileye sensor
- tzakrajs 9y ago
  The self driving should be capable of mitigating this risk by adapting to systems like these in general.
  - fixermark 9y ago
    Problem is, that greatly complicates the problem-solviing process for SD vehicles---you're now trying to build atop a model of the world provided by the vendor, with its own faults and quirks (and you probably need an agreement with the vendor to see their code / dissect their model, and now your release schedule depends on their release schedule, etc.). It also makes Uber dependent upon Volvo's buy-in on the project, and that arrangement is fraught in a world where we don't yet know if SD cars are going to damage the business of traditional auto manufacturers 50 years out---Uber wants to be in a place where if a vendor relationship sours, they can pack up and change chassis without a complete overhaul of the SD logic. If your SD system is supposed to be solving 100% of the same problem anyway, it's best to gut the software you don't control and work with the hardware as directly as you can. The notion of building atop a lower-level abstraction makes for great academia but doesn't work with industrial realities.
    - tzakrajs 9y ago
      Thanks for the explanation and I agree that it won't be as easy to build SD for auto braking systems that change from chassis to chassis, but this is what I think people want. People want to have fail safes from different vendors.

kwhitefoot 9y ago

Automatic emergency braking is quite definitely a backup system. If it ever trips in my car it is because the primary system, that is me, has failed to brake in time.