8 ms·
> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if
by a_t48 7y ago
> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary).
> Why were there no alarms going off at 5.6 seconds when the vehicle was confused!!!??
That sort of depends on the specifics of how their obstacle prediction and avoidance works - having a fuzzy view on what exactly something is at 5.6 seconds out is probably ok. The important bit is that it notices that the obstacle is moving out to the road and should stop it. Classification is not needed to avoid objects. The key words here are actually "at each change, the object's tracking history is unavailable" and "without a tracking history, ADS predicts the bicycle's path as static" which is a horrible oversight.
>> That triggered what Uber called “action suppression,” in which the system held off braking for one second
> This is borderline criminal negligence.
Yeah, this is. Even though it was too late to swerve, there was still enough time to slow down and attempt to reduce the speed of impact. _This_ is probably where an alarm should have fired - the car is now in a really unsafe state and the safety driver should absolutely know about it.
Disclaimer - Up until a few months ago I worked at a major (non Uber) self driving car company.
- gambiting 7y ago>>The important bit is that it notices that the obstacle is moving out to the road and should stop it The problem here is that if I'm doing 70mph+ on the motorway and I see a plastic bag flying in my path, the correct course of action isn't to brake hard or panically try to avoid it - but a computer cannot tell a difference between a soft plastic bag and something more rigid. If the only classifier is that "there's an object in my way, I need to stop" then that's also super dangerous. In fact, there are cases where unfortunately the correct course of action is to keep going straight and hit the object - any kind of small animal you shouldn't try to avoid, especially at higher speeds - yes they will wreck your car, but a panicked pull to the left or right can actually kill you. As a human you should do this instinctively - another human = avoid at all cost, small animal = brake hard but keep going straight, plastic bags = don't brake, don't move, keep going straight. How do you teach that to a machine if the machine can't reliably tell what it's looking at?
- simion314 7y agoMy conclusion is that until you have hardware and software to classify objects as well a human you don't even consider testing on public roads. I can't understand the optimism that with such bad hardware and software but with tons of training it will magically get better then a human, Also we need some tests before we let anyone put his self driving car on the streets, should I be able to hack my car and npm install self driving and then test on your street ?
- a_t48 7y agoI don't have a great answer to that - I don't know Uber's capabilities and I can't really speak to my previous company's capabilities either. Sorry! Edit: I will say that "size and relative speed of the object" is an excellent start, though.
- sizzle 7y agoWhy aren't they driving the cars with the driverless software in learn mode, scaled up to many human drivers that are training the autonomous classifiers edge case dataset e.g. human driver stops for this particular shaped object (person in darkness entering street) in 99% of cases, so we stop as well. (As opposed to not stopping for a floating grocery bag where it applies the opposite rule to keep driving based off recorded human driving behavior) Also, we can detect human gait already can't we? There has to be better signifiers for human life that we can override these computer vision systems with to prevent these tragedies. What about external safety features, such as an external airbag that deploys on impact, cushioning the object being struck if the system senses immediate collision is unavoidable. Some food for thought for anyone working in this space, would love to hear your thoughts and ideas.
- a_t48 7y ago> Why aren't they driving the cars with the driverless software in learn mode, scaled up to many human drivers that are training the autonomous classifiers edge case dataset e.g. human driver stops for this particular shaped object (person in darkness entering street) in 99% of cases, so we stop as well. (As opposed to not stopping for a floating grocery bag where it applies the opposite rule to keep driving based off recorded human driving behavior) This basically is being done already - recorded data is hand labeled as ground truth for classifiers. "human in darkness entering the street" might be a rare case, though. Humans in the dark on the sidewalk should be pretty common. > Also, we can detect human gait already can't we? There has to be better signifiers for human life that we can override these computer vision systems with to prevent these tragedies. I'm not a perception expert - it looks like in this case the victim was mostly classified by LIDAR capture. I don't know if the cameras were not good enough for conditions or if their tech was not up to the task. This is sort of a red herring though - even if the victim was classified as "bicycle" or "car" she still should have been avoided. > What about external safety features, such as an external airbag that deploys on impact, cushioning the object being struck if the system senses immediate collision is unavoidable. Some food for thought for anyone working in this space, would love to hear your thoughts and ideas. Why not do this on all cars?
- freewilly1040 7y agoA bit off topic, but what is your overall view of the self driving sector, aside from the more egregiously irresponsible players?
- a_t48 7y agoLong term optimistic. Short to medium term is fuzzy for me. :)
- cameldrv 7y agoMy guess from the NTSB description is that their tracker had object category as part of the track information, and that detections of one category weren't allowed to match a track of another category. This is useful if, say, you are tracking a bicycle and a pedestrian next to each other. When you go to reconcile detections with tracks, you know that a bicycle isn't going to suddenly turn into a pedestrian, this rule helps to keep you from switching the tracks of the two objects. Unfortunately that also means that when you have a person walking a bicycle that might be recognized as either one, this situation happens, and they initialize the velocity of a newly tracked object as zero. Even this though shouldn't have been enough to sink them, because the track should have had a mean-zero velocity, but with high uncertainty. If the uncertainty were carried forward into the planner, it would have noted a fairly high collision probability no matter what lane the vehicle was in, and the only solution would be to hit the brakes or disconnect. Furthermore, if you've initiated a track on a bicycle, and there's no nearby occlusion, and suddenly the bicycle disappears, this should be cause for concern in the car and lead to a disconnect, because bicycles don't just suddenly cease to exist. They can go out of frame or go behind something but they don't just disappear.
- jacquesm 7y ago> Classification is not needed to avoid objects. Taking your disclaimer into account, that is a very wrong assumption. If you classify an object wrongly that means your model will not be able to properly estimate what the object's capabilities are, including its potential acceleration speed and most likely trajectory from a stand-still. And that information will come in very handy in 'object avoidance' because a stationary object could start to move any time.
- scoot 7y ago> Disclaimer - Up until a few months ago I worked at a major (non Uber) self driving car company. That'd be a disclosure. (When you disclose something, it's a disclosure.)
- dragonwriter 7y ago> That'd be a disclosure. It's a disclosure that is (among other things) disclaiming neutrality, so also a disclaimer.
- scoot 7y ago> It's a disclosure that is (among other things) disclaiming neutrality That being the purpose of a disclosure, my point stands. What was yours?
- dragonwriter 7y agoMy point is that your pedantry claiming as wrong a reference to one completely accurate description because another description, which is not exclusive of the one provided, also applies is, itself, wrong, and, further, that the personal insults based on it are, —as well as being unnecessary, infantile, off topic, and counter to the HN guidelines—also unjustified.
- mysterydip 7y agoHere's a quick way to get safety in your collision avoidance system up to spec: randomly choose coders of the system to be the objects avoided during tests.
- bsaul 7y agoI’d add the managers and VCs that probably put all the pressures in the world onto the developers, that end up doing hacks just to meet the unreasonable expectation of the employers.
- m0zg 7y agoI know you jest, but it's very easy to not reliably kill anybody. Just don't let the car move at all, or stop every time it detects something as large as a mosquito. What's hard is to not kill anybody and actually drive.
- Khaine 7y agoDamn straight. These fuckers (and their executives) should be the ones whose lives are on the line, not innocent people on random roads.
- roenxi 7y agoYou might be able to literally count on two hands the number of accidents where we have this level of transparency into the thought processes and sensory data of the 'driver'. And control over what happens next time. There is no doubt that the engineering data gathered here and in other accidents is going to contribute to a massive (truly massive, unspeakably massive, enormously massive - all such adjectives are appropriate) reduction in both actual deaths from car accidents and statistical deaths from the huge amount of time people waste driving. The big picture is so overwhelmingly positive that even if the engineers were purposefully running people over to gain testing data it would still probably be a large net win for greater society. Thankfully there is no call for such reckless data gathering. If anything, punishments in this case should be much more lenient than normal rather than breaking out the cruel and unusual measures.
- Piskvorrr 7y ago
- chooseaname 7y ago> Classification is not needed to avoid objects. The key words here are actually "at each change, the object's tracking history is unavailable" and "without a tracking history, ADS predicts the bicycle's path as static" which is a horrible oversight. Was it an oversight or were they having trouble with the algorithm(s) by keeping the object history and decided to purge it?
- a_t48 7y agoPure speculation incoming: probably a completely unintended bug. Classify as static, static objects shouldn't have tracks, clear the history. Your object is reclassified as dynamic, whoops, we already cleared the history, oh well.
- vilhelm_s 7y agoStatic is not a classification; in this case the classification was cycling between "unknown", "vehicle", and "bicycle", and each time the classification changed the history was reset. Then each detected object can be either static or be assigned a predicted path, but unknown objects without a history are always treated as static. In any case, the history clearing seems to be a bug, they updated the software to keep the position history even when changing the classification.