10 ms·
"According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavio
by jerf 8y ago
"According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior."
If you can't give your "self-driving software" full access to the brakes because it becomes an "erratic driver" when you do that, you do not have self-driving software. You just have some software that is controlling a car that you know is an inadequate driver. If the self-driving software is not fully capable of replacing the driver in the car you have placed it in, as shipped except for the modifications necessary to be driven by software, you do not have a safe driving system.
- jjoonathan 8y agoSo, let the two software systems fight with each other during emergencies and hope it somehow results in something positive? That sounds like a terrible idea.
- JorgeGT 8y agoOn the contrary: the Volvo system is designed not to fight but to override any other input, including the human driver. In any other Volvo car currently on the road that detects an obstacle, the Volvo system will override the driver and bring the car to a stop even if the driver insists in hitting the obstacle: https://www.youtube.com/watch?v=oKoFalJiazQ https://www.youtube.com/watch?v=oKoFalJiazQ Independently-working override-capable systems are the base of engineering redundancy safety. See airborne collision avoidance systems (ACAS), which will automatically and forcefully steer an aircraft to avoid a collision if necessary: https://en.wikipedia.org/wiki/Airborne_collision_avoidance_system?oldformat=true https://en.wikipedia.org/wiki/Airborne_collision_avoidance_s...
- jjoonathan 8y agoIf the Uber engineers shared your confidence that it would play out that way, they would not have disabled it. Redundant safety systems are a great idea, evidently Uber needed more of them, and integrating with the Volvo system might have been a reasonable option. It's silly to suggest that the integration would necessarily have been trivial, though. That's what I'm objecting to.
- JackCh 8y agoYou're being very charitable in your assumption that Uber 'engineers' gave it much thought at all. (Were any Professional Engineers even involved in this? Or was it just a bunch of "software engineers"?)
- jjoonathan 8y agoHowever much thought they gave it, I guarantee that everyone in this comment train has given it much less. It wasn't a trivial decision, their choice is not prima facie evidence of negligence.
- JackCh 8y agoAgain, very charitable. Your assumptions are baseless, seemingly motivated by little more than your trust that incredible incompetence and negligence is rare in the real world.
- malandrew 8y agoAgain, not at all charitable. Your assumptions are baseless, seemingly motivated by little more than your mistrust that in the competence of people that have been actively working on this problem, many for years.
- JackCh 8y agoHe said that he can "guarantee" Uber engineers gave it careful consideration (or at least more than people in this thread), but in truth he can do nothing of the sort. There are a plethora of examples where engineers and "engineers" didn't give a problem careful consideration and it got people killed. There is no rational basis for guaranteeing this is not one of them. There is no "guarantee" that uber "engineers"/engineers put more thought into this than "their system is inconvienent, tear it out" or "we don't need their system because ours will be better, tear it out." Nobody can guarantee Uber engineers were not stupendously negligent until the investigation is complete. Anybody who thinks they can guarantee that has an irrational basis for thinking they can provide such a guarantee.
- bmelton 8y agoI don't have experience with self-driving cars, Uber or Volvo safety systems, but my Jeep's backup collision prevention often slams the brakes if I'm backing up and the camera detects potholes, oil stains, or irregular painted lines in the parking lot. If any of the systems are vulnerable to any such false positives, and equipped to enable emergency braking to avoid them, even on the highway, it's not hard to imagine why they might be disabled, especially during testing phases. I think it's fair to say that at this early stage in product development, there's probably no 'always right' answer for how to handle a given obstacle without considering locality, driving speed, road conditions, likelihood of false positives and negatives, etc.
- wilsonnb 8y agoThey shouldn't be testing on public roads and highways then.
- repler 8y agoYeah mine does that too. I wish they had a sensitivity option like they do with the forward collision sensors. Hey there's an idea for Uber, maybe instead of disabling the forward collision system entirely they could just decrease the sensitivity to lessen false positives (like in our Jeeps)?
- ubernostrum 8y agoTCAS systems won't automatically take over. Instead they audibly issue an order to the flight crew, describing the action that must be taken, and the flight crew must take the action. Instructions from TCAS are near absolute in their priority. If ATC says to do something different, you ignore ATC and do what TCAS says. If the pilot in command says to do something different, you ignore the pilot in command and do what TCAS says. If somehow God Himself is on your plane and tells you to do something different, you ignore Him and do what TCAS says. Compliance with TCAS is non-negotiable, and the Überlingen disaster[1] is the bloody example of why it's that way. Self-driving/autonomous-car systems need to have a similar absolute authority built in. If Uber disabled theirs because of false positives, it's a sign Uber shouldn't be running those cars on public roads. [1] https://en.wikipedia.org/wiki/2002_%C3%9Cberlingen_mid-air_collision https://en.wikipedia.org/wiki/2002_%C3%9Cberlingen_mid-air_c...
- JorgeGT 8y agoAFAIK Airbus has a TCAS-capable AP/FD, certified at least for the A380, which can automatically fly RAs by itself: https://www.skybrary.aero/index.php/Autopilot/Flight_Director_(AP/FD)_TCAS_capability https://www.skybrary.aero/index.php/Autopilot/Flight_Directo...
- vkou 8y agoEvery time you drive a modern car, you fight for control with a computer driving system. Somehow, it works out.
- dmoy 8y agoUnclear why this is being downvoted. Modern cars have all sorts of computer driving systems, e.g. ABS. If you learned to drive without it, you still think to pump even though the computer does a better job if you just slam on the brakes. The point is that so far most (all?) of these computer systems don't have a failure mode where it kills pedestrians, because their scope is very limited.
- sdfgdfhjdgj 8y agoBecause one doesn't "fight" with ABS not wanting to break. It will break when you hit the brake. Same with power steering: it won't steer into a different directorion. You don't have to "fight" with it.
- jackhack 8y agoYou might be troubled to know that the guidance control systems on modern spacecraft and many aeronautics subsystems work this way -- at least three redundant systems have access to sensor data and each make independent decisions about the course of action, if any, needed. In short, they vote, and majority wins.
- marcosdumay 8y agoThere is no "fight with each other". If any system says "break", the car breaks. If none says that, the Uber system has control.
- dogma1138 8y agoThe irony is that this is 100% the what MobilEye said in their in their model demo I think 3-4 years ago. Their CEO said that the regulators cannot regulate all breaking conditions and should should only test for false negatives exactly to prevent this. He stated that dealing with flase positives is going to be in the best interest of every manufacturer and the easiet thing to do is to disable the breaks completely but that would be fairly easy to detect in any structured test or a test drive. Any thing else would be a problem because you would be stacking tolerances and worse creating test cases with a clear conflict of interest where the interests of the car maker and the regulator align.
- jpeg_hero 8y agoThis comment made my brain hurt. I think you have something interesting here, that’s why I read it four times and still didn’t understand. Please re-write this for clarity.
- diffeomorphism 8y ago> If the self-driving software is not fully capable of replacing the driver in the car you have placed it in, ...,you do not have a safe driving system. Nobody disagrees with you and that is explicitly the reason why a human is on board, so I am not sure what you are arguing against.
- Clubber 8y agoMaybe he was just making a statement.
- Obi_Juan_Kenobi 8y agoIs that enough? I think it's very clear that a human driving a normal vehicle is different from a human sitting at the wheel of a self-driving (semi-self-driving?) vehicle. You simply cannot expect a human to remain as engaged and attentive in such a passive situation. It's baffling to me that they chose to deactivate emergency braking without substituting a driver alert. If the false-positives are so frequent as to render the alert useless (i.e. it's going off all the time and you ignore it) I don't think these vehicles are suitable for on-road testing. You can lean fairly heavily on the 'still dangerous, but better' argument in the face of 40,000 US vehicle fatalities each year, but there are limits.
- azinman2 8y agoExcept it’s patently clear that is a flawed concept. Here’s a prime example where a human, not paying attention because the car has been successfully driving for a while, is given 1-2 seconds to emergency stop the car. That is not enough time to process what’s going on and take over. Even 10 seconds is likely not enough.
- masklinn 8y agoWorse, under Uber's regime the one human had to deal with both emergency situations (without warnings from the system) and instrumentation feedback (without a HUD), so the intended operation of this test was that the safety driver spends half the time looking down and away from the road.
- budu3 8y agoI presume that expectation was that the human driver was suppose to be fully engaged to take evasive actions if and when required. What jumps up at me if that fact that the system does not alert the driver when his/her intervention is required.
- dmoy 8y agoI'm not sure it matters, because going from disengaged to fully engaged and actually breaking to a stop in 1.3s is not happening for an average human. By the time the system alerts the driver, the ped is already dead.
- estebank 8y agoBut an attentive human would have noticed the pedestrian around the same time the system was confused, ~6s before impact. Enough time to swerve and stop.
- chopin 8y agoAt 6 sec before impact the driver for sure would assume the system will notice and brake in time. Until the driver realizes that the automated system will NOT brake its very likely too late. Automatic systems which rely on humans braking in time have no place on the street, imho. This may be different for lane assist where malfunctioning is more obvious and leaves more time for intervention. Although even in this case the latest Tesla accidents may tell a different story.
- jiveturkey 8y ago> If you can't give your "self-driving software" full access to the brakes because it becomes an "erratic driver" You misunderstood. The Uber software had sole control of the brakes (plus the human of course). The Volvo factory system was disabled so that it didn’t have negative interaction with the Uber system. Your mistake is understandable. The article was poorly written, perhaps due to a rush to publish, as is the norm these days. Even if the NTSB report was unclear, that doesn’t excuse clumsy reporting. If you’ve ever done significant mileage in a car with an emergency braking system you probably have experienced seemingly random braking events. The systems favor false positives over false negatives.
- anon1385 8y ago>You misunderstood. The Uber software had sole control of the brakes (plus the human of course). The Volvo factory system was disabled so that it didn’t have negative interaction with the Uber system. That's not how I read it, or how any of the journalists who are reporting the story are reading it. Uber disabled the self driving software's ability to do an emergency stop when it detected it was going to crash. The Volvo system is separate and also was disabled when the car was in self driving mode. https://www.bloomberg.com/news/articles/2018-05-24/uber-self-driving-system-saw-pedestrian-killed-but-didn-t-stop https://www.bloomberg.com/news/articles/2018-05-24/uber-self... >Sensors on an Uber SUV being tested in Tempe detected the woman, who was crossing a street at night outside a crosswalk, eventually concluding “an emergency braking maneuver was needed to mitigate a collision,” the National Transportation Safety Board said in a preliminary report released Thursday. >But the system couldn’t activate the brakes, the NTSB said. That's the only reading that makes sense to me, otherwise why did the car fail to attempt to stop when it detected the pedestrian and knew it was going to hit them?
- eslaught 8y agoIs that true? A cousin comment appears to indicate Uber had a parallel emergency breaking system and that is what failed to trigger: > At 1.3 seconds before impact, the self-driving system determined [...] Unless this is exceptionally poorly worded, it certainly sounds like self-driving system was the one doing the determination. https://news.ycombinator.com/item?id=17144741 https://news.ycombinator.com/item?id=17144741