8 ms·
Watch a Tesla Fail an Automatic Braking Test and Vaporize a Robot Pedestrian
- trabant00 6y agoIt's almost like software has bugs. Even at magical company by the internet's most loved wizard. Who wants to bet their astro-turfing squads aaa, sorry, PR teams will be all over this downvoting and disliking on every social media platform.
- ethbr0 6y agoThe converse conspiracy theory holds equal validity: that China might have an incentive to generate poor test results for non-Chinese autonomous / electric vehicles. An industry they've publicly stated is a national priority.
- zpeti 6y agoThis definitely seems like a legitimate test. Especially with the music and everything.
- zingplex 6y agoGiven that it is similar to the results found by the AAA tests last year [1], I have no problem believing these findings are legitimate and reproducible. Nonetheless, it would be helpful to know the speed at which the vehicle was moving and other such factors. [1] https://www.aaa.com/AAA/common/aar/files/Research-Report-Pedestrian-Detection.pdf https://www.aaa.com/AAA/common/aar/files/Research-Report-Ped...
- NikolaeVarius 6y agoAh yes, the random twitter account from China. The best of journalistic sources
- nendroid 6y agoOnly the best journalistic sources can be trusted even when it's a live video because the entire thing can be completely fake! I mean who knows, if it's not from a "best" journalistic resource than it could be some random dude in his moms basement spending millions of dollars to create several computer generated videos of a Tesla failing to stop for a dummy. People these days will believe anything.
- jsight 6y agoIt doesn't take millions, it just takes a willingness to hold down the accelerator as an override.
- vernie 6y agoAh well you see Tesla's system is so sophisticated that it knew that it was a dummy and not a real pedestrian.
- drno123 6y ago1. The test may be fake, and Tesla driven manually. 2. Maybe Tesla uses not only cameras but also radar sensors. A blow-up dummy is transparent for radar. Real human is not. So it is OK to drive through balloons.
- hexane360 6y agoCar radars aren't sensitive enough to pick up pedestrians anyway. They're useful for detecting metal vehicles, and pretty nothing else. If you mean lidar, dummies are not transparent to those wavelengths.
- drno123 6y agoThis is not true. Car radars do pick pedestrians, but some car radars (blind spot for example) filter that out in post processing.
- enragedcacti 6y ago1. the entire point is to test the emergency braking features which are active by default. 2. I would definitely prefer a false positive where tesla brakes for human shaped balloons crossing the street than the potential for plowing through a human because of a bad radar reading. this video is obviously not unbiased but here you can see a Tesla plow through a dummy that is obviously not made of balloons https://youtu.be/tkAtRw51jGY?t=81 https://youtu.be/tkAtRw51jGY?t=81
- jsight 6y ago> 1. The test may be fake, and Tesla driven manually. The second video had in car footage clearly displaying a visual warning and playing a collision warning sound. Because of concerns over phantom braking, these can be overridden by the accelerator pedal. I wouldn't call it fake, but it is exaggerated.
- kbos87 6y agoThe IIHS does all sorts of autonomous braking test scenarios involving pedestrians - https://youtu.be/TJgUiZgX5rE https://youtu.be/TJgUiZgX5rE A pedestrian darting out from behind a car is known to be a tricky scenario, but some systems handle it at least some of the time. This scenario looked like the pedestrian was unobstructed and traveling in a predictable direction, so it’s pretty unnerving if it’s real. Long story short, even in Teslas, even just autonomous braking and crash avoidance is a great extra safety layer, but far from something that should be relied upon. Tesla is reckless for selling it as something that can be relied upon to successfully pilot a car.
- toomuchtodo 6y agoThis is a common issue across all vehicles with radar AEB systems, and the problem isn't vehicle manufacturers but the general public in the belief that these systems are more robust than they are (pedestrian collision avoidance). To your point about Tesla, they vigorously communicate that you (as do other manufacturers) as the driver are responsible at all times. We can go roundabout like we always do about this, but regulators have continued to allow these systems to be marketed and sold, and they're the authority. Why are we blaming automakers for garbage drivers? The driver is who you sue when they hit a pedestrian. > To understand the strengths and weaknesses of these systems and how they differ, we piloted a Cadillac CT6, a Subaru Impreza, a Tesla Model S, and a Toyota Camry through four tests at FT Techno of America's Fowlerville, Michigan, proving ground. The balloon car is built like a bounce house but with the radar reflectivity of a real car, along with a five-figure price and a Volkswagen wrapper. For the tests with a moving target, a heavy-duty pickup tows the balloon car on 42-foot rails, which allow it to slide forward after impact. > The car companies don't hide the fact that today's AEB systems have blind spots. It's all there in the owner's manuals, typically covered by both an all-encompassing legal disclaimer and explicit examples of why the systems might fail to intervene. For instance, the Camry's AEB system may not work when you're driving on a hill. It might not spot vehicles with high ground clearance or those with low rear ends. It may not work if a wiper blade blocks the camera. Toyota says the system could also fail if the vehicle is wobbling, whatever that means. It may not function when the sun shines directly on the vehicle ahead or into the camera mounted near the rearview mirror. > The edge cases cover the gamut from common to complex. Volvo's owner's manuals outline a target-switching problem for adaptive cruise control (ACC), the convenience feature that relies on the same sensors as AEB. In these scenarios, a vehicle just ahead of the Volvo takes an exit or makes a lane change to reveal a stationary vehicle in the Volvo's path. If traveling above 20 mph, the Volvo will not decelerate, according to its maker. We replicated that scenario for AEB testing, with a lead vehicle making a late lane change as it closed in on the parked balloon car. No car in our test could avoid a collision beyond 30 mph, and as we neared that upper limit, the Tesla and the Subaru provided no warning or braking. https://www.caranddriver.com/features/a24511826/safety-features-automatic-braking-system-tested-explained/ https://www.caranddriver.com/features/a24511826/safety-featu...
- GhostVII 6y agoI dislike Tesla's misleading FSD claims, but this is a pretty unfair test imo. They timed the movement of the pedestrian so it was only in front of the car when it had less than 10 feet to stop, of course it will hit it. I don't think we should have the expectation that automatic braking is always able to predict someone's movements, regardless of how good your system is cars need more than 10 feet to stop. And the car actually did stop, it just couldn't stop before hitting the pedestrian because it had no distance. Edit: the second test is pretty bad for Tesla though, the dummy was stationary
- VikingCoder 6y agoYou and I see very different things when we watch that video. There's absolutely no doubt in my mind that a competent human driver would have avoided hitting the dummy.
- jib 6y agoThe dummy in no way looks human in the context I’d expect a self driving system to have. Humans don’t have a way to move their legs back and forth that way while remaining stationary (unless on a treadmill I suppose). You can infer that it is meant to represent a human, and thus likely to move across a crossing, but there’s no reason to expect a computer to read it as a human right? The most obvious characteristic is distinctly not human (Legs don’t work that way), so it looks like some weird windmill, or one of those hot air moving men.
- simongr3dal 6y agoI wouldn’t my car to drive into a wheeled mannequin rolling onto the street either.
- wrsh07 6y agoDo you think it's fine to hit things that don't correctly get categorized as human? Like maybe someone on crutches?
- 6y ago
- loceng 6y agoThis test is so poorly done it feels like it must have been done for propaganda or bad PR. Aside from the legs barely moving, the arms not moving - e.g. it doesn't look like a pedestrian the AI would be modeling from - it's driving out of context through parking lot lines. Edit to add: the guidelines for using Autopilot, irrespective of what the name insinuates to you, requires the driver to be paying attention at all times and takeover if it's not doing what you expect; "BUt iT shOUld BE PerFeCT" attitude is dismissing one's personal responsibility - if you don't want to have to pay attention then don't drive with Autopilot.
- onion2k 6y agoIt could be a 100-ton concrete block in the road. You don't want your car to drive in to it regardless, no matter if it's a person or not, and no matter what the lines on the road look like. You car driving itself in to anything is bad.
- TFortunato 6y ago"Sorry you lost a leg, but to be fair, you don't walk like the type of people in our training set"
- mhh__ 6y agoThis is sort where jaywalking originates from i.e. people are getting run over so make it illegal to cross
- djxfade 6y agoShould it matter? It's still an obstacle that it shouldn't be hitting
- deleted 6y ago[deleted]
- abeppu 6y agoEven if it doesn't move its arms like a prototypical walking human, shouldn't the car still stop, rather than ram into an unidentified large obstacle / skateboarder / pedestrian carrying something? If it's driving in a parking lot rather than a busy city street, shouldn't that have meant it can also safely swerve away from the obstacle in almost any direction?
- onion2k 6y agoEven if you totally ignore the fact it's person-shaped, a self-driving car should still stop if it's going to drive in to something.
- andrenotgiant 6y agoImagine you are on the highway in self-driving mode and there's a plastic bag flapping around on the road ahead. Probably best NOT to stop in that scenario. It's the millions of "edge cases" that humans are pretty decent about making quick decisions on that make self-driving cars really scary to me.
- onion2k 6y agoProbably best NOT to stop in that scenario. The car should slow down for that scenario. It doesn't know if the situation is 100% safe, so it should apply the brakes and steer to avoid a collision using data from around the car to determine the safest course. Driving through a hazard is only an option for a self-driving car if there is literally no other option. Cars are not people and they're not capable of making decisions in the same way people do. They should be taking the safest route all the time.
- Karunamon 6y agoIt should not stop in that scenario because there is no hazard. Wild application of brakes is just as dangerous as not stopping in a hurry when necessary.
- deleted 6y ago[deleted]
- onion2k 6y agoI've not driven a Tesla, but I'm going to guess they have different levels of braking between "not braking" and "wild application of brakes", so when I said it should "slow down" I obviously meant something in the middle.
- patd 6y agoAnother similar test where the Model 3 did break: https://youtu.be/cMiZa3HgRVE?t=140 https://youtu.be/cMiZa3HgRVE?t=140
- john_miller 6y agobreak != brake!
- falcolas 6y agoNon-deterministic results aren't exactly a point in favor of Tesla's auto braking system.
- deegles 6y agoIs there a name for how the FSD feature is structured to never be responsible for accidents? Tesla has so many disclaimers about how you always have to keep your hands on the wheel and be alert, so when accidents do happen they're blamed on the driver. Catch-22 maybe?
- falcolas 6y agoFraud? Telsa tried to use the "His hands weren't on the wheel, look, he got an alert" argument in the NTSB investigation into an AP fatality. It was later proven that the alert was some 7 minutes prior to the accident.
- foobiekr 6y agoYou should consider whether Tesla considers Autopilot engaged if it was disengaged, say, 10ms before the collision. It's pretty clear from their long history of weasel-wording that the answer is no, but they also aren't saying.
- eplanit 6y agotl;dr the "robot" is a mannequin, and "vaporized" is merely being struck.
- notsuoh 6y agoGiven your tldr which presumably summarizes the relevant parts of the story, is it your take that this is a nonstory because the autonomous Tesla simply hit the pedestrian at a high rate of speed rather than "vaporizing" them?
- nabla9 6y agoI wonder if this is a case where camera+radar sensor fusion gets confused. Radar in Tesla's is supposed to be only supplementary sensor but it's there and contributes for decisions. >Something made of wood or painted plastic, though opaque to a person, is almost as transparent as glass to radar. https://www.tesla.com/blog/upgrading-autopilot-seeing-world-radar https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...
- wrsh07 6y agoThis should be concerning to people since there are often wooden barriers around construction sites
- nabla9 6y agoWith black box deep learning models it's hard to know. DL models learn complex representations but fundamentally what they do is just interpolate between examples. Throw something out of distribution at them and they behave strangely. For example (speculation): 1. the system knows how to stop when it recognizes stationary radar transparent structures (traffic cones or wooden structures) 2. the system has experience with plastic bags on the road and learned to drive trough them without stopping. 3. the visual system has learned to stop when it sees walking humanoid figure. The previously unseen rivalry between 1+2+3 leads to delayed decision.
- tibbydudeza 6y agoElon is working on a new "baggo meato sensor" using Neuralink and German shepherd brain tissue to "sniff" if a object is a "real" human being or a fake imitation.
- kovek 6y agoHere is a longer video, seems like other cars are able to break before hitting the pedestrian. Also, there is no "still" pedestrian, as some point out. https://m.weibo.cn/detail/4551307186670198#&video https://m.weibo.cn/detail/4551307186670198#&video Still rooting for Tesla to figure this out!
- xiphias2 6y agoAlthough I agree with others that Tesla is overselling the current FSD capabilities, it seems that the deep learning community is figuring out 3D video perception (Tesla taking a big part of it, though I'm not sure how many papers they publish)..it improved like crazy last year. Just look at a few videos from two minute papers: https://www.youtube.com/user/keeroyz https://www.youtube.com/user/keeroyz
- me_me_me 6y agoWhat a time to be pedestrian :D
- tghw 6y agoMany of the comments here seem to be missing the point. It is an object moving into the path of the car, and if the safety systems work as advertised, it should avoid that object. Whether it's a person, shopping cart, bike, go kart, etc. doesn't matter.
- kbos87 6y agoExactly - that object could have just as easily posed a mortal threat to the driver of the Tesla, yet it didn’t stop.
- mhh__ 6y agoIt's also worth saying that the human brain can almost instantly do a fight or flight calculation as to whether it's a moose or a plastic bag floating by taking into account the motion, and it's reflective properties etc. This task clearly is very very hard to do, but tough shit: if you can't safely control a megajoule of kinetic energy coming down the road your computer shouldn't be given the benefit of the doubt.
- Mightlymoose 6y agoThe way the 'Pedestrian' is walking looks more like a Nazi marching and thus this may be a feature of Tesla and not a bug
- notsuoh 6y agoDid you create this account specifically to make this unfunny comment? It and a duplicate of it are your only comments at all. What's your usual username, just post it under that.
- Mightlymoose 6y agoThe 'Pedestrian' is walking more like a Nazi soldier marching. This actually might be a feature and not a bug
- dathinab 6y agoA similar article (a bit older but not? Chinese based tests): https://insideevs.com/news/377427/video-tesla-model-3-failed-aaa-test/ https://insideevs.com/news/377427/video-tesla-model-3-failed... An interesting insight was that pedestrian what most early detected by the Tesla but it just decided to brake only just before hitting them...
- akmarinov 6y agoGuys, guys, this is a thing with a lot of cars Here's BMW, Mitsubishi and others failing at the same thing: https://www.youtube.com/watch?v=7Y8JG7kepwc https://www.youtube.com/watch?v=7Y8JG7kepwc The only reason you don't hear about it, is because hating on Tesla gets clicks.
- what_ever 6y agoI don't see BMW, Mitsubishi calling their system AutoPilot and their CEOs making all kinds of outlandish claims.
- logicOnly 6y agoTesla marketing claims they have a self driving car. Tesla legal says you couldn't possibly think Tesla has a self driving car.
- seattledev 6y agoNo man you don't get it, Elon Musk is a FAILURE and everything he touches is GARBAGE and his cars are the worst thing EVER /s
- alexc05 6y agoWATCH: code running in debug scenario fails to run successfully.
- seattledev 6y agoWATCH: system designed to stop for vehicles fails to stop for something not a vehicle
- mcguire 6y agoUnrelated question: I'm reading this with Firefox on Android. None of the Twitter links work, reporting "Cannot Complete Request "Additional information about this problem or error is currently unavailable." Is the problem Twitter or Firefox?
- notsuoh 6y agoThis happens consistently with me. Reload the window where Twitter is appearing and it always loads on the second try.
- sp332 6y agoTry updating Firefox? I've been having this trouble with Nightly Firefox and Twitter for a few days. On desktop, clearing Twitter-related site data fixed it, but on mobile it was only fixed in this morning's build.
- r053bud 6y agoYeah this is why I avoid Teslas on the highway when I'm driving. Their owners are more often than not on their phones not looking at the road while autopilot has their lives (and others) in its hands.
- tdhz77 6y agoAs a former Tesla owner I can tell you we feel the same about humans controlling cars. On the highway between Baltimore and DC there was 6 of us on autopilot, we had FSD and we all kept a safe distance and passed slower cars for about 40 miles. It made me realize if we all had the same tech in cars in terms of safety and OS, we could be safer than human control and we were.
- libertine 6y agoHonest question, if the driver accelerates does it overdrive the AI? In other words, if the autopilot would break, but driver accelerated, what would the car do? Also, what SHOULD the car do? I'm asking this because we have no idea how the tests were done: we're given a video of a moving Tesla hitting a dummy and breaking after. I don't know how people can take any conclusions from this information, other than what is shown. Disclaimer: I don't own any Tesla stock, neither I have any Tesla products.
- GrinningFool 6y agoAbout as interesting as watching a unit test fail without context.
- IshKebab 6y agoGiven that Tesla are pre-selling full self driving that will never exist, what kind of financial liability are they exposing themselves to? Do their T&Cs say "Oh btw we might never deliver this. No refunds!"?
- seattledev 6y agoWow, an automotive news source heavily entrenched with the traditional manufacturers editorializes the hell out of an article about Tesla. What are the odds.
- mhh__ 6y agoAs opposed to Tesla fanboys fawning over anything Musk says regardless of reality?
- whamlastxmas 6y agoYou're saying it's okay for a journalistic outfit to be biased and write hit pieces that can have real impacts on the markets because Elon Musk also has an army of fans that defend him?
- jmpman 6y agoMaybe this is a software localization? https://slate.com/news-and-politics/2015/09/why-drivers-in-china-intentionally-kill-the-pedestrians-they-hit-chinas-laws-have-encouraged-the-hit-to-kill-phenomenon.html https://slate.com/news-and-politics/2015/09/why-drivers-in-c...
- shaftway 6y agoFor the first video I don't understand how this could have worked to begin with. There's no way my Tesla would have allowed triggering autopilot in a parking lot with so many lines.
- amai 6y agoTeslas deep learning auto pilot is so smart, that it understands this is just a dummy. So no need to hit the break :-/