32 ms·

Tesla Model 3 Drives into Overturned Truck in Apparent Autopilot Failure

• risyachka 6y ago
  These accidents don't really matter as long as the ratio of accidents it avoided to accidents happened is greater than one. People make fuss when autopilot fails, critic that its based on camera etc etc. But when accident is avoided, no one will even notice.
• anlqo 6y ago
  Looking forward to the day that they find out that LIDAR actually damages human eyes. Yes, I know that the current doctrine is that it does not, but these things tend to get reversed after a decade.
• dmode 6y ago
  I have a Model 3. It still does phantom braking. Elon’s FSD claims are laughable and this story is just another data point
• DoingIsLearning 6y ago
  Is the Tesla 3 using stereo cameras _only_? I thought most of these ADAS systems had some form of sensor fusion, combining multiple systems like: Stereo camera + front radar + long range US ... etc. This is a scenario that would be safely avoided with an entry-level AEB system, is there something special about Tesla's design that I am missing? edit: typos/clarity
  • markus92 6y ago
    They do have ultrasound (12 sensors they claim), but that doesn't help for detecting static objects iirc.
  • tim333 6y ago
    I think they have a front radar. The trouble with those is differentiating stationary stuff like bridges from stationary stuff like overturned trucks.
    • DoingIsLearning 6y ago
      I think negating low-hanging bridges is one of the scenarios OEMs specifically calibrate for with auto manufacturers. It's not rocket science it's a tuning/calibration processs for that specific vehicle's dimensions and layout. Front radar calibration is not a new problem and is well understood in the auto industry. These accidents really raises the question, assuming they do have a front radar, what is Tesla doing so differently that allows a competitor entry-level AEB system to outperform them? To me it seems that NHTSA, TUV, etc. should ban Tesla's auto-pilot on public roads until they can show evidence of their safety.
• Erlich_Bachman 6y ago
  Many of the comments seem to jump onto the "autopilot doesn't work, we told you so"-train. I think it is important to remember and be clear that no autopilot system will be 100% safe, nor is any human driver 100% safe. What is interesting is looking at statistics of how often these accidents happen, and comparing that statistic to human drivers. There are other issues like the way we calculate statistics: for instance we shouldn't just calculate "miles driven with autopilot engaged", without looking at how it was disengaged, what would have happened if it wasn't disengaged at that time, whether that would lead to an accident, etc. There are issues, yes, but some commenters are acting like they think that this accident is what happens every time there is an obstacle, and as if the autopilot does not have functional obstacle detection at all. This is not the case. This is an exception, a failure in the obstacle detection, not the rule. The fact that this news is getting so much traction is a testament to that. If the autopilot was not detecting any obstacles, we would have news articles like this every week.
• pintxo 6y ago
  Forget the truck, there seems to be a person (the truck driver?) standing some 50m in front of the truck, waving the oncoming cars to the side. And the Tesla seems to completely ignore the fact that a human is standing basically right in his planned trajectory. Missing the truck, fine. But why is the system not reacting to a human on the highway? Humans standing on dedicated highways is likely a 99% signal for trouble ahead.
• cjdell 6y ago
  I have owned a Tesla Model 3 in the UK for almost a year now. The Auto Pilot is only useful on dual carriageways and straight roads and still very frequently suffers from "phantom braking". It should be obvious to any half decent driver that concentration is not optional. Having said that, the Auto Pilot has saved me from a couple of near misses: - It has detected the car in front braking suddenly and reacted before my brain had even registered. - It has nudged the car sideways when a car (in my blind spot) moved too close. At the moment it is a driving aid, a second set of eyes, not a driver replacement. It is useful as that, but Tesla (as much as a love them) could really do to be more honest in the marketing of their "Full Self Driving" packages. I expect it will get substantially better, but not in the time frame the Musk says it will.
  • globular-toast 6y ago
    > Having said that, the Auto Pilot has saved me from a couple of near misses I'm interested to know, have you ever been involved in any collisions on the road before owning the Tesla? How long have you been using the road?
    • cjdell 6y ago
      Only 1 accident that was my fault (I was young and stupid). Been driving for 17 years and have had near misses before. I don't know for sure if I would have crashed if it wasn't for Auto Pilot, but it's nice to know that it's there, even if it makes mistakes from time to time. The other factor is that I'm driving more now that I have the Tesla, which probably isn't too surprising (at least until the lock down).
      • globular-toast 6y ago
        Whenever I hear about these safety features saving people from near misses I always wonder how they've got this far driving without dying. For example, lane keep assist. If you can't stay in a lane you can't drive. It's such a basic requirement. So I can only assume that people are deviating from their lanes simply because these safety features exist.
• ksec 6y ago
  460 Comments Right now. Lots of criticism of AutoPilot and Marketing. "5" ( very brief) Mention of Elon Musk. I am not sure what to make of that. May be people feeling what he has achieved on SpaceX meant he could get away with whatever he said / promised on Tesla. Or may be I am just hyper critical of Him ( With Respect to Tesla only ).
• ap46 6y ago
  Their under body sonar & image processing should have picked it up before the attempted braking...hope they put out full postmortem.
• fastball 6y ago
  Is it not necessary to confirm with a source besides the driver (e.g. Tesla) that Autopilot was actually on? I feel like "I had Autopilot on and it didn't work" is a pretty convenient excuse if it's not on and you have an accident all on your own. Because to me that is entirely plausible, yet we're all operating under the assumption (and having lengthy discussions about) this "failing" of Autopilot.
• bequestry 6y ago
  Yikes, the airbag didnt go off... can the NTSB look into this case?
• ajankelo 6y ago
  Am I the only Tesla owner that finds the blame on autopilot a little hard to believe? My car never reaches the point where I feel that I don’t have to pay attention.
  • johnghanks 6y ago
    I'm blown away at how trusting of AP and FSD people are. I've only used AP once or twice in the 6 or so months I've had my 3 and it consistently scares the shit out of me. It's not _bad_ per se, but it drives like a teenager and that has resulted in a fundamental distrust of the system on my end.
• Animats 6y ago
  Didn't even slow down. Humans often brake late, but seldom fail to brake at all. It's grossly irresponsible of Tesla to ship a system that can't detect big, solid obstacles. This happens over and over. So far, they've hit a crossing semitrailer, a street sweeper, a car at the side of the road, a fire truck at the side of the road, and two road barriers. This has been going on for five years now, despite better radars and LIDARs.
  • sgt 6y ago
    Doesn't this take us back to the discussion around the word Autopilot? Tesla sells it as driver-assist, no more autonomous than an autopilot system in a Boeing 737. Good for following the obvious path, and otherwise needs input from the actual pilot.
    • c22 6y ago
      It seems like the obvious path would be around solid obstacles. I've never driven a Boeing 737, but will the autopilot really fly you directly into an obstacle with no warning at all? My understanding was that these vehicles are equipped with radar and collision avoidance systems.
      • floatingatoll 6y ago
        Generally, non-military aircraft avoid collisions using radio broadcasts from various beacons (their own and those of other planes). Actual radar systems for detecting non-broadcasting obstacles are, per Wikipedia, generally only found on military aircraft. So if you define "an obstacle" as something that is not a plane broadcasting a beacon, then, yes, a Boeing 737 would probably run directly into it if it were somehow in the path of the autopilot.
      • s_y_n_t_a_x 6y ago
        Yes it will, autopilot just keeps the heading and the altitude. There may be other sensors onboard that help with detections (it is a lot more expensive after-all) but the autopilot control system does no such thing. Even with all the sensors onboard, the 747 would most likely run into something without assistance, in fact I've seen a few cases of this on Air Disasters.
• bra-ket 6y ago
  i think autopilot is a misnomer in this case, it should have been called brainless driving assistant on a serious note, it's not that another edge case was missing in training data but what we call "common sense" is fundamentally missing in neural network based approach, and no amount of parameters, model permutations and gpu power would compensate for that
  • tylerhou 6y ago
    This isn't a neural network edge case/common sense issue; this is an issue with the radar sensors that Tesla is using for driver assist. Specifically, radar is noisy — it will constantly detect non-existent "stationary" objects in front of the car. Driver assist systems therefore ignore any objects which are stationary on the road — if it didn't, then the car would stop erratically.
    • Traster 6y ago
      You know what wouldn't have had this issue? Lidar.
    • sandworm101 6y ago
      >> Driver assist systems therefore ignore any objects which are stationary on the road If true, that is totally unacceptable. Any mountain road can have fallen rocks. Any city road can have a child's toy be stationary in the road. And EVERY road can at any time have a live person, a fallen motorcyclist/bicyclist, lying crumpled up and stationary in the middle of the road. Stationary objects, whether on radar, camera or both cannot be ignored. If the radar 'sees' such an object for more than a heartbeat or two, the car should take action. That's what any reasonable person does when they aren't sure what they are looking at.
      • renewiltord 6y ago
        Why do you say this? I'm at the wheel so I can always act. The driver assist makes this easier to do by helping out but I'm still here at the wheel and so I don't need perfection, I just appreciate the help.
• djstein 6y ago
  should the car have slowed down by itself? yes. was the driver not paying attention and did not have their hands on the wheel? obviously. it is driver assistance, you agree to the terms and conditions, you still have to DRIVE the car.
  • DanCarvajal 6y ago
    But it's Autopilot, the car should auto pilot itself.
    • dlivingston 6y ago
      That's a misnomer. Just as autopilot in an airplane requires active attention by a pilot, so too does autopilot in a Tesla require active driver attention.
      • CamperBob2 6y ago
        Ask 99 out of 100 people, and they'll say an autopilot "makes the plane fly itself." I'm 100% on board with Tesla and most of Musk's other efforts, but calling this system "Autopilot" was negligent. Are you (speaking to the audience at large, not to the OP) an engineer at Tesla? Did you speak up when they decided to call this thing "Autopilot"? Why or why not?
        • toomuchtodo 6y ago
          > and they'll say an autopilot "makes the plane fly itself." I don't believe in tailoring your marketing for your most ignorant customers. If you ask a Tesla advisor during a test drive, they will very clearly state the car does not drive itself (ask; I've tried more than once to see if they'd say something other than company stance). It is extremely clear in the vehicle's manual and other documentation put front and center for the driver that Autopilot is meant to be an assist, and the driver is responsible at all times. Don't turn on the feature if you're going to ignore the warning messages and legalese. Example in-vehicle copy: https://i1.wp.com/electrek.co/wp-content/uploads/sites/3/2017/02/autopilto-update-2.jpg https://i1.wp.com/electrek.co/wp-content/uploads/sites/3/201... https://www.garyshood.com/tesla-autopilot-features/forward.jpg https://www.garyshood.com/tesla-autopilot-features/forward.j... ("Similar to the autopilot function in airplanes, you need to maintain control and responsibility of your vehicle while enjoying the convenience of Autopilot in Model S")
• yumraj 6y ago
  It seems that none of the airbags deployed (mentioned in the story), per: https://twitter.com/jsin86524368/status/1267423319495606272 https://twitter.com/jsin86524368/status/1267423319495606272
  • floatingatoll 6y ago
    That tweet and its replies note that the driver was uninjured and that the airbags might not have been required. For example: > Looking at the HQ video, it looks like the impact speed wasn’t that bad and the crumpling of the truck decelerated the car nicely. The passenger compartment is fine.
  • teraflop 6y ago
    It's hard to judge from the video, but the car doesn't appear to have been going very fast, and it looks like the side of the truck crumpled a bit to absorb the impact. Airbag inflation is a violent, potentially injurious, last-ditch effort to prevent fatalities. If the airbags didn't inflate but the driver walked away from the accident, then not activating was probably the safer option.
  • deleted 6y ago
    [deleted]
  • FireBeyond 6y ago
    Early days of airbags had fairly simple deployment requirements. Current vehicles have a lot more, in terms of impingement, impact direction, instant and sustained G-forces, etc. When teaching EMTs we use airbag deployment as an indicator of potentially serious mechanism, but the reverse isn't true - you can't automatically dismiss an accident as minor because there was no deployment in an airbag-equipped vehicle.
• kevinsundar 6y ago
  There are so many ethical issues[1] with Tesla marketing autopilot as self driving given it makes mistakes as grave as this. [1]https://arxiv.org/abs/1901.06244 https://arxiv.org/abs/1901.06244
• deegles 6y ago
  Under what conditions could a crash unequivocally be Autopilot’s fault? This looks like it will be blamed on the driver.
  • leetbulb 6y ago
    No matter the name of the feature, I think it can only be blamed if it had totally removed control from the driver. This accident would've been avoided had the driver been paying attention. Cars are heavy death machines, not apps, and I have zero sympathy for people who take them for granted.
  • SiempreViernes 6y ago
    Well, Tesla puts disclaimers that their "assistant" demands the driver to always be paying attention so they would accept blame could only if the system completely ignores correct user input. I don't know if there is any instance where it would completely ignore user input and Tesla agrees the users wasn't at fault.
  • gpm 6y ago
    If autopilot created a situation in which a crash was inevitable with no warning to the user. E.g. if it suddenly grabbed the steering wheel and swerved right resulting in the car spinning out in only a few hundred ms for no good reason. Even if autopilot caused a crash such as above, you still have to run a cost benefit analysis of whether or not it prevented more crashes than it causes. Considering the failure rate of humans it's ok for autopilot to have a non-negligible failure rate, and it doesn't really matter if those failures happen in the same places as the human failures.
    • deegles 6y ago
      Wouldn't Tesla say "you should have had your hands on the wheel, therefore it's not Autopilot's fault"?
      • gpm 6y ago
        Hands on the steering wheel doesn't mean affirmative control. That's why the example includes putting the car in an unrecoverable scenario faster than an attentive person can react.
• randcraw 6y ago
  Assuming autopilot was indeed engaged, this seems to be clear evidence that cameras alone are insufficient to perceive big objects in a car's path OR judge its distance to them. This car traveled in a straight line in near-perfect weather for at least 10 seconds toward a large motionless object blocking its lane and the next one, but still overlooked the danger. Imagine if this had happened in bad weather or at night, where risk of failure is substantially greater. This degree of incapability implies that vision alone is not only inadequate to perceive obstacles ahead that should be obvious to any human, but profoundly inadequate.
  • birdyrooster 6y ago
    Nah dude, everyone trusts Elon Musk even though he lies repeatedly. Amazing that someone so unethical could come from a family mining blood Emeralds in South Africa. But alas, keep buying his vehicles because he totally cares about human lives.
    • close04 6y ago
      What's worse is that it didn't detect the person that was clearly on the car's lane.
  • svachalek 6y ago
    Humans do it with vision alone, do they not? This seems to be a failure of processing, not data.
    • thefounder 6y ago
      How do you think the objects are detected with "vision" by autopilot? Are you considerig classifying all the objects in the world in all the possible situations so that that dumb computer can avoid them? So far that seems the plan.
      • skellington 6y ago
        You don't need to train with every object in the world to train in the general concept of road occluders. Autopilot users are ALWAYS supposed to watch the road and take control in unusual circumstances. Like an autopilot system in an airplane, it will fly you straight into a mountain if you give it that course and don't monitor where you are.
• morningcovfefe 6y ago
  This is not a self-driving car and it is not cruise-control (that demands you pay attention to the lane ahead of you). This is a savant 8 year old driving your car. Seems to behave fine in common cases but if you relax you might well die if an uncommon case comes up that this child has no basis to handle. I personally would rather have an actual 8year take the car, since then I would have the correct amount of attention on the road out of well founded fear for the limitations of the child's abilities
  • colinmhayes 6y ago
    Do we have any evidence that shows 8 year olds can't drive if given enough practice? I mean as long as they can see and reach the pedal I'd say driving is simple enough that they could do it if there aren't any distractions.
    • MertsA 6y ago
      Forget 8 year olds, my 3.5 year old can drive a tractor through the woods no problem. We obviously only let him steer and have someone else on the throttle, but he can keep it centered, turn around, avoid obstacles, etc just fine. Kids can handle bipedal locomotion just fine, it's not like they don't have the ability to make rapid decisions to avoid obstacles or correct their position. Steering is no different, it just takes some practice and experience to adapt to it. It's the higher functions that children aren't suited to do.
    • teraflop 6y ago
      There was an incident just a few weeks ago where a 5-year-old in Utah took his parents' SUV for a joyride and managed to make it to the interstate before getting pulled over. Based on the police footage, he wasn't doing a very good job of maintaining his lane, but he apparently managed to avoid hitting any stationary objects, so... go figure? https://www.youtube.com/watch?v=C-W-ySng2t8 https://www.youtube.com/watch?v=C-W-ySng2t8
    • throwaway0a5e 6y ago
      Special interest groups spent the 1930s-50s hand wringing over young kids driving (which was a pretty rare thing anyway) and got a bunch of laws written so clearly some kids were driving just fine at some point. It wasn't decades later when seat-belts came along and alcohol got taken seriously that the roads got appreciably safer so it's not like they were dangerous enough to have an affect despite being rare.
• joshuahedlund 6y ago
  I remember early on when Tesla was touting some kind of crashes/deaths per mile average or something compared to the US average... Anyone have the current numbers on that?
  • alexslobodnik 6y ago
    Directly from Tesla <<In the 1st quarter, we registered one accident for every 4.68 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 1.99 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 1.42 million miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 479,000 miles. Total overall miles and crashes were significantly reduced in this quarter.[0]>> [0] https://www.tesla.com/VehicleSafetyReport#:~:text=Q1%202019,every%201.76%20million%20miles%20driven https://www.tesla.com/VehicleSafetyReport#:~:text=Q1%202019,....
    • tqi 6y ago
      On thing to keep in mind is that the nature of the miles driven when autopilot engaged are not the same as the ones without autopilot (ie mostly highway miles vs city street driving), and likely also different from the miles driven without active safety features.
      • reitzensteinm 6y ago
        Yes. It's a poor argument and Tesla knows it. My intuition was always that autopilot + human would be safer, and the only thing that made me question that was the soggy toast Tesla provides as "proof". There would be privacy concerns doing an apples to apples comparison, but people could opt in.
      • gnramires 6y ago
        Agreed, those numbers alone don't convince it's safer. Also, personally I would have a complication: would I be satisfied with being safer than average? I don't know if I'm average. I tend to pay attention and never have driven under influence (alcohol or w/e), or when sleepy. So a little better than average would not bring me peace of mind. I think some kind of situational statistic would be relevant here; maybe they could have some part of the fleet as controls with AP disabled (and offer some kind of reward), then the overall accident rate can be compared better.
• pengaru 6y ago
  Tesla's marketing, particularly of autopilot, has been criminally misleading IMHO.
  • a4wtaw4tawtawt 6y ago
    Idiots eat it up. They also worship a man-baby, so critical thinking isn't really their thing.
• NicoJuicy 6y ago
  I somehow agree with Musk that a self driving car should be able to work with only cameras, as humans do with their eyes. If there are 2 cameras, depth can be calculated ( eg. 2 eyes) and Tesla is in the best position to achieve this ( lots of data). Additional sensors make the system way more complex and most of them have the same issue: it doesn't work in the worst weather conditions. They should however greatly check why the system failed and how they can improve depth vision with their cameras. The failing of the system in broad daylight is a severe issue/fault. And I'm sure they will improve this. Until full auto pilot, which is still a long way to go, the AutoPilot is driver assistance and driver's should be aware.
  • fctorial 6y ago
    if (car.aboutToCrash()) { car.stop() }
    • NicoJuicy 6y ago
      If there is someone behind them, it would be better to try drive around the obstacle.
  • rainyMammoth 6y ago
    Interesting argument. Why doesn't a plane fly like a bird? Why don't we have submarines that swim like fishes? (for the sake of clarity, yes I'm being sarcastic to demonstrate how stupid that argument is. Humans have eyes but also got a brain that is way more powerful than any AI that we have today)
    • NicoJuicy 6y ago
      Because the technology isn't/wasn't there and is "good enough" as it is. But it doesn't compare to the efficiency of millions of years of evolution. Eg. Feathers for flying + flexible bone structure and absorbing skin for water pressure. I will react with the question: although we can dive and fly. Wouldn't it be better, if we adapt to nature's evolution? Eg. Put on a feather- suit, jump start the engine to get off the ground and then float to the destination and only let the engine kick in when we need to go higher/land? Riding on earth's natural wind and sparing fuel resources ( going off topic here) On topic: ML/AI should be able to datafy all related information that human eyes perceive. And those datapoints need to be handled by every situation that occurs. The article clearly shows that this is not the case yet. While a brain has more adaptability, Tesla should be able to counteract this with more data of the vehicles that drive around.
• jannyfer 6y ago
  When something similar happened in 2016, I read that Autopilot will tune out bright objects against a bright sky. https://www.forbes.com/sites/dougnewcomb/2016/07/18/elon-musk-tweets-tesla-will-make-changes-to-autopilot/ https://www.forbes.com/sites/dougnewcomb/2016/07/18/elon-mus... I’m imagining that a low-dynamic-range camera may see certain tunnels/underpasses’ exits as bright white objects blocking the road. The above article refers to a now-deleted Musk tweet that says something similar.
  • simonklitj 6y ago
    Your comment made me remember that multiple Tesla engineers quit and spoke out about the insufficient sensors a couple of years ago: https://www.technologyreview.com/2017/08/25/149475/some-tesla-engineers-think-autopilot-isnt-safe/ https://www.technologyreview.com/2017/08/25/149475/some-tesl...
• xiphias2 6y ago
  The most important part for me is that the driver is uninjured. Sure, autopilot made a huge mistake, but Model 3 with Autopilot still has a very good safety record.
  • rconti 6y ago
    Are you suggesting that the car is intelligent enough to only plow into massive stationary objects if they'll crumple adequately so as to prevent driver injury? A better argument would be about overall occupant safety vs driver-controlled vehicles per mile travelled.
    • xiphias2 6y ago
      No, I was quite worried about the driver being dead, and the article hasn't written about it, or hid it somewhere. It's just basic human empathy that in an accindent safety of people should be no. 1, and then safety of the system should be addressed.
• elihu 6y ago
  The car did eventually detect the obstacle and brake prior to impact, otherwise it would have been a lot worse. What seems weird to me from the video is that there was a person beside the road (perhaps the driver of the overturned truck) waving at cars to stop, and the Tesla didn't detect the pedestrian and slow down, even though he/she appeared to be standing on the edge of the car's lane. It's hard to tell for sure, but it looks like the pedestrian would have been hit if he/she hadn't stepped out of the way.
  • ProAm 6y ago
    >Tesla didn't detect the pedestrian and slow down It's because pedestrians are not an object normally located on an interstate. They need to stop calling it Autopilot. I want to see the data if the drivers hands were on the wheel at the time of impact, and the eye tracking sensor data for the 30 seconds prior to impact.
    • reitzensteinm 6y ago
      Detecting a pedestrian where they are not meant to be is exactly when you should slow down. That's a weird choice, so I assume they were having false positives and couldn't leave the detection on.
      • X6S1x6Okd1st 6y ago
        NN are known to have problems with generalizing. There are likely very free Few examples of pedestrians on the high way
        • visarga 6y ago
          Tesla is using a data-engine approach where they source rare situations of interest from the fleet and include them in the training loop. Basically their whole approach is relying on finding and plugging holes. I bet pedestrian detection and accident detection on the highway was covered a long time ago.
• ashtonkem 6y ago
  Apparently the airbags didn’t deploy, which is a serious problem.
  • LeoPanthera 6y ago
    If the car was already braking, and it hit a soft enough obstacle (in this case, the roof of the truck), the G force may not have been enough to trigger them, which would be correct behavior.
    • ashtonkem 6y ago
      Possible, the first images of the wrecked vehicle seemed severe enough to warrant airbags.
      • rconti 6y ago
        Thankfully, airbags do not deploy based on what us internet commentators deem "seems severe", but, instead, based on data from accelerometers.
    • maxdo 6y ago
      yes, the driver remains unharmed.
    • irjustin 6y ago
      Which is incredibly surprising given the damage in the picture, but always possible I suppose.
• philipov 6y ago
  Did the passenger die?
  • milansuk 6y ago
    From article: "Luckily, nobody was seriously hurt, but the Model 3 was trashed and covered with what one source called “sauce”"
• nihil75 6y ago
  Who was it that said (and I'm paraphrasing) - "Self-driving is easy, you could build an AI yourself that will work 90% of the time. It's writing an AI that works 100% of the time that's difficult". He built a car that drove cross country. It's not "cameras vs. LADAR", it's about software that needs to work flawlessly. I love "Lean" development process, and "just get it out there", but I wouldn't write banking software that way. Or autopilots for that matter. A blatant disregard for human lives.
  • legohead 6y ago
    100% is impossible. It's how many 9s are you willing to accept. And LIDAR would add a nine or two, but Musk is just being stubborn.
    • justapassenger 6y ago
      He's stubborn, but he also realistically I don't they have any choice. Retrofitting all the cars they sold with FSD, with set of LIDAR sensors (or refunding those purchases) would likely bankrupt the company. Buying a car with FSD HW years before we even know what's needed to get FSD, is like buying a PC, spec'ed to run Duke Nukem Forever, in 1997.
      • gonesilent 6y ago
        As a 5 day new Tesla driver playing with auto pilot you can just feel the thing is over worked. The road I live on HWY 36 in California was a joke on autopilot. Car can't do it has to slow well below speed limit. Every other turn it tries to drive off the road and gives up steering. Also roads with cracks that are black with pavement sealer confuse the car to no end. The car is color blind and the vision system gives it no time to react.
• cashsterling 6y ago
  Stuff like this is completely expected of nonlinear, high dimensional controllers (and neural networks used for object detection and decision anlaysis fit into this category)... NN's are not fit for high-predictability, high-safety-factor control. I wonder if, in fact, we cannot construct a NN controller with the level of unusual object detection and edge-case scenario recognition required to match an average human drivers capability to recognize and react appropriately to unusual situations. I mean to say that I think this is something like an NP-hard problem... to make it one-log better you need 100x the data and 100x the NN dimensionality. Model complexity explodes... I also think we are spending a lot of time and effort solving the wrong part of the transportation problem. Cars have a thermodynamic problem for society --> a 1500kg car to move 1-2 80kg people is very energy inefficient and resource intensive to manufacturer. We need remote work, more home delivery options (e-truck/vehicles), greater incentive for electric bikes, and better/more public transportation. my 0.02...
  • fctorial 6y ago
    I think it was trained (or learned) to avoid the rear of vehicles, and slow down when it sees it. The rooftop doesn't look anything like the rear.
    • gambiting 6y ago
      Surely a Tesla has a front radar to measure distance though, no? If it's rapidly approaching to an object that's not moving, who cares what it looks like?
      • oldgradstudent 6y ago
        Can it actually distinguish reliably between an overnight truck and a manhole cover, an overpass, and an overhead sign? These are far more common occurrences. Reportedly it can't. https://www.wired.com/story/tesla-autopilot-why-crash-radar/ https://www.wired.com/story/tesla-autopilot-why-crash-radar/
        • 6y ago
• fortran77 6y ago
  Does the budget "Tesla 3" have the same sensors and autopilot functions as the full-featured "Model S" or are they stripped-down?
  • joenathanone 6y ago
    Same sensor suite.
• stcredzero 6y ago
  How about a limited LIDAR that only looks straight ahead? (Perhaps in a 5x5 degree wedge?) That's likely to be far cheaper than a full LIDAR set, and would provide useful data about a barrier in the vehicle's path.
  • zucker42 6y ago
    Somehow I doubt this problem is as simple as "add a LIDAR" considering there's probably at least one smart person at Tesla who has thought of that idea. Sure you might be right, but my impression is that the 1 in a million cases will still exist.
    • tpmx 6y ago
      There would still be that Musk person dictating no LIDARs, so, no.
      • WrtCdEvrydy 6y ago
        Adding LIDAR would make the millions of cars on the road a liability. You'd have to make those people whole and basically recall the cars. If you can 'make it work' with cameras, you save millions.
        • tpmx 6y ago
          Seems like they've sold about 1 million cars to date. https://www.theverge.com/2020/3/10/21172895/tesla-one-million-cars-production-model-y https://www.theverge.com/2020/3/10/21172895/tesla-one-millio...
  • anon102010 6y ago
    I agree - and I think you're seeing new products in the lidar space with a focus on cost constraint (with more limited resolutions). I really do hope tesla adopts at least 1 or 2 of these cheaper options when they get cheap enough, my guess is a 50% of lidar would get you 80% of the benefits and fix a lot of these long tail edge case issues around drive-able areas.
• mindfulplay 6y ago
  This is getting a bit ridiculous. Silicon Valley mindset being applied to critical safety focused products is really bad. I hope they regulate the crap out of Tesla and God I hope comma.ai whose founder seems even more batshit crazy.
  • tim333 6y ago
    Comma.ai's system actually has an eyeball tracker that I think stops things if you are not looking where you are going which would likely prevent this kind of crash.
• mindfulplay 6y ago
  The level of respect that someone like Andrej Karpathy is jarring: he should publicly apologize instead of being all saintly (oh look ma, I built a toy with 6 cameras, I am so proud of myself) while talking about deep learning or whatever crackpot stuff they are cooking over thinking they are solving world's "problems". Pick your "heroes" properly HN crowd.
  • rcMgD2BwE72F 6y ago
    Was Autopilot even activated?
• ChrisClark 6y ago
  Every radar based distance following system has a disclaimer that it ignores stationary objects. It will be great when it's purely based on vision.
• solarkraft 6y ago
  What's the accident/distance driven now? How does it compare to a human?
• deleted 6y ago
  [deleted]
• tqi 6y ago
  Do ML models for self driving cars start from an assumption that it is safe to proceed forward unless it identifies an obstacle, or does it assume that it is unsafe to proceed forward unless it identifies that the path is clear?
• mixmastamyk 6y ago
  A few years ago I found it hard to believe driver assist systems were doing all sorts of massive complicated (potentially buggy) processing instead of "large object directly ahead ---> slow/stop." Folks gave all sorts of reasons why that wasn't good enough. I'm going to reiterate here that I don't want anything more complicated than very simple forward collision prediction (resulting in slow down) aka "Automatic Emergency Braking" in any car I purchase.
• Plutonsvea 6y ago
  Ah, an outlier. I don't see any significance to this story other than being a $TSLA trigger word... I've seen many people, including truck drivers get hurt as a result of relying too much on technology (Lane assist, auto-headlights, etc). This is why they're all labelled as an assistive technology. The driver should have seen a gigantic truck blocking his entire lane, and though it was a failure of autopilot to not detect it, autopilot shouldn't be receiving all the blame for something that is inherently not its job to do 100% perfectly.
  • mixmastamyk 6y ago
    If assist technology can't avoid an object the size of a barn from 100 meters, I would argue it didn't do its job even 1% perfectly.
  • MattGaiser 6y ago
    Claim a car is “full self driving“ and it’s news when it can’t drive itself.
    • ggreer 6y ago
      Tesla doesn't claim that their current cars are full self driving. They claim that future software will allow FSD on some models sold today.
• rowawey 6y ago
  Teslas have been marketed unsafely and promote unsafe behavior. They cannot see many classes of stationary objects, especially at high speed. And because of Dear Leader's over-reliance on "magic bullet" IP investment, they're not going to change how it works voluntarily.
• rck 6y ago
  Haven't seen anyone mention the human that almost gets hit. If the person standing in the lane (the truck's driver maybe?) hadn't stepped out of the way, the Tesla would have run him over without slowing down... Looks like multiple clear perception system failures.
  • LeoPanthera 6y ago
    Or, autopilot was not engaged. I haven't seen any evidence yet that it was turned on at all.
    • dagmx 6y ago
      Regardless, the car should have emergency breaking. As it stands, it seems like Tesla struggles at novel stationary objects right in front of the car. It seems like their training is more geared to Known object types in front, and arbitrary objects on either side
      • rcMgD2BwE72F 6y ago
        Yet, in most third-party tests and evaluations, Tesla's AEB get better restults than the rest of the industry (with a 94% safety rating: https://www.euroncap.com/en/results/tesla/model-3/37573 https://www.euroncap.com/en/results/tesla/model-3/37573). Mmh...
• argonaut 6y ago
  This demonstrates the superiority GM's Super Cruise approach. They have interior cameras that make sure the driver is looking forward (you're allowed to look away for brief periods). Tesla only forces you to keep one hand applying some pressure to the wheel - you could easily be on your phone with your other hand. GM's approach is much more relaxing and prevents accidents like this. That being said, there's no reason Tesla can't easily adopt this approach in the future.
• bronco21016 6y ago
  Somewhere along the line the marketing of Autopilot fell apart. As an airline pilot when I hear the word autopilot I think of a system that generally drives the aircraft along a programmed routing and vertical profile. It is most definitely not a set it and forget it device. Training repeatedly stresses that someone must be monitoring the actions of the autopilot at all times and it’s taken very seriously as it can rapidly place you in a very undesirable state. Somewhere along the way Tesla Autopilot became marketed as a fully self-driving system freeing us to tune out. I think the criticism about the capabilities of the system could be toned down some but we most definitely should be critical of what capabilities are advertised and what instruction is given for the person operating the system. I think it’s very likely that an Autopilot like system could make driving significantly safer but only if we figure out the human interaction aspects and fully and truthfully communicate the capabilities of that system.
  • vagab0nd 6y ago
    They used "Auto Cruise" in the 80s. https://youtu.be/hfRiedxPQhs?t=362 https://youtu.be/hfRiedxPQhs?t=362
  • 6gvONxR4sf7o 6y ago
    As an airline pilot, how long do you typically have to respond to an autopilot issue? Is it seconds? Split second?
    • bronco21016 6y ago
      That is an aspect that is admittedly different from a car. It really depends on the phase of flight. In cruise there are rarely instances that an immediate reaction is needed. In the terminal environment though where you’re closer to the ground and other traffic it can definitely be split second. Autopilots will incorrectly intercept approach guidance and fly through the course or altitude when you’re low to the ground or blow through the lateral course into a parallel runway’s course. That’s the most frequent failure I’ve encountered. I’ve also seen it just completely lose its mind on autoland at low altitude requiring intervention. Point is split second decisions are needed during critical phases of flight. Driving a car on autopilot is likely going to require that level of vigilance during the entire drive unless we get highway systems that present hyper controlled environments where self-driving cars have higher reliability. This can kind of be likened to why cruise is less critical in an aircraft. You’re at high altitude, traffic is exclusively controlled by ATC (unlike low altitude where there is traffic not under ATC control), and the aircraft are all required to operate on autopilot and have collision avoidance systems where the aircraft communicate with each other (TCAS). Aviation has dealt with the integration of these systems for the last 5 decades and the human/machine interface. There have been many failures and successes and I think it would be a great place to start for figuring out how this works with cars.
• blacksoil 6y ago
  I'm very not comfortable with camera-only and no-sensor approaches in self-driving cars. While they can train a neural network with millions or even billions of sample images from real-life tests, I do think they're still bound to fail in some corner cases. Remember FB's face detection that turned out failed to detect faces of people of certain races? I think there's still some possibilities that NN in self-driving cars would fail to recognize some corner cases such as a grandma crossing a street with particularly unique combination of clothes and colors.
• jayd16 6y ago
  Does Tesla use autopilot for emergency break assist as well? This seems much worse than what's in a lot of other cars in terms of automatic breaking.
• mysterEFrank 6y ago
  That's thankfully far out of distribution. Why doesn't Tesla get more slack for incidents like this? Uber had to shut down its self-driving lab for almost a year when one of its vehicles failed and killed a pedestrian.
  • sukilot 6y ago
    Because Uber hired a human driver that ran over a pedestrian while doing paperwork when they should have been driving. Tesla, for all of its overselling autopilot, didn't the tell these drivers to take naps and play video games while on autopilot.
  • 6gvONxR4sf7o 6y ago
    At the time, Uber had critical mass of bad press for many reasons.
  • jcranmer 6y ago
    Why should Tesla get any slack? This particular circumstance looks like it should be within the typical guidelines for Automatic Emergency Braking systems to detect and react to--the sorts of systems which can be found in consumer models of cars. Uber shut down its self-driving lab in part because the circumstances were such in the realm of "the car should be capable of handling it" that excess caution was warranted in figuring out why a very easy case was missed.
• siliconunit 6y ago
  Continuous laser telemetry should be always available and active, anything bouncing back at certain speed and distance starts autonomous braking... No need to identify anything..
• m0zg 6y ago
  That's called a "false negative", ouch.
• m0zg 6y ago
  I get how a computer vision system missed this unusual situation, but how did a _radar_ miss it?
• fnord77 6y ago
  > For any human driver paying even the slightest bit of attention, this accident is almost an impossibility, assuming the driver had the gift of sight and functional brakes. No. A friend of mine was driving home up 101. There was a large boat sitting in her lane. She thought her brain was playing tricks on her and did not believe there was a boat sitting on the highway. And she crashed into the boat.
  • moralestapia 6y ago
    Well then ... A friend of mine was driving home up 101. There was a large boat sitting in her lane. She thought her brain was playing tricks on her but eventually figured out there was actually a boat sitting on the highway. She was able to avoid the boat. That's the problem with rumors.
• whoisthemachine 6y ago
  I see a lot of mention of neural networks in relation to Autopilot. However, it's hard to imagine that those pursuing automatic driving wouldn't combine neural networks with normal old rules and safety thresholds (ie "just brake" if a new, sufficiently large, mostly uniform object is filling much of the sensors). Is this not the case?
• robszumski 6y ago
  Why don't they have 6 super cheap single beam lidars in some sort of pattern to detect this type of thing? It doesn't have to be a giant spinning sphere on the top of the car. Small moving prism in one axis?
  • WrtCdEvrydy 6y ago
    Because Tesla's CEO says he wants a camera-only solution.
• lumberingjack 6y ago
  Worst part is no airbags worked so whole car was FUBR
• 627467 6y ago
  Unpopular opinion: Anything that promotes (directly or indirectly) less driver engagement with driving should just be banned from civilian roads. I'm sure one day when everyone is being driven by allsensing selfdriving vehicles road deaths will have massively decreased. Until then, technologist just will have to learn how to get there in much more controlled environment. A legal waiver or T&Cs should not be the only thing 'protecting' the next self driving victim.
  • jmpeax 6y ago
    > Unpopular opinion: Anything that promotes (directly or indirectly) less driver engagement with driving should just be banned from civilian roads. I take it you turn off your phone and the car radio when driving, and also tell all of your passengers they must also turn off their phones and maintain strict silence.
• darwinwhy 6y ago
  I did a search for "lidar" in this thread and found nothing. But this seems like the kind of obstacle that lidar would detect every time? I'm not an autonomous vehicle engineer, so can anyone more qualified comment on whether this crash is a symptom of Tesla's decision to avoid lidar?
  • deleted 6y ago
    [deleted]
  • MPSimmons 6y ago
    The number of obstacles that LIDAR could prevent hitting, which Autopilot would not pick up, over the average life of a car, is very small. Whether LIDAR would be worth the money is, I suppose, an academic exercise. You could calculate the number of Teslas involved in such crashes, and divide the price of a Tesla by the price of LIDAR, to decide if it is worth it (I suspect not, at current prices), but ultimately, things will happen even with LIDAR. It isn't a silver bullet. I'm on my second Tesla. I'm totally biased. It has saved me from inattention several times. I probably wouldn't actually have crashed any of those, but I might have. Regardless, I'm glad I have it, and I am safer with it than without it. Sure, I could screw up and crash on autopilot, but it would be easier to crash without autopilot.
    • Dylan16807 6y ago
      If the LIDAR is the critical piece to get the car above level 2 automation on the highway, then that's worth quite a bit of money.
      • booi 6y ago
        afaik LIDAR is not useful in even minor precipitation (rain/snow/fog). If the car is dependent on LIDAR at highway speeds then this needs to be addressed. Being dependent on cameras is similar to a human situation and is (hopefully) no worse.
        • Dylan16807 6y ago
• tlapinsk 6y ago
  It seems pretty clear that while the AutoPilot technology may be cool or fun, it is nowhere near ready for full-self driving as Tesla has marketed it as. Using cameras + radar + neural networks may not be enough - no matter how cool the underlying software stack is. Similarly, I also question other self-driving car manufacturers (Waymo, Cruise, Uber) approaches using a Lidar/Radar/Camera suite. The long tail of the problem is immense and going to take years to solve with the current technology. To me it's clear that a new set of technologies that are more dynamic and responsive going into unknown situations is going to be needed to make it "foolproof" (if that is even possible at all). Disclaimer: This is coming from someone who naively thought full-self driving cars would already be on the road by now back in 2015
• kirillzubovsky 6y ago
  Why is this stuff getting upvoted? All we see is a Tesla that rammed into an obstacle on the highway, with evidence of anything else. We don't know that it was on autopilot, and the driver in this case would say anything to try and get his insurance money back. The car started to break when it saw an inevitable collision, but otherwise, it's pure speculation.
• accurrent 6y ago
  Personally I believe we should have fail safe back ups. A radar/lidar would have prevented such an accident from happening. You can't prove that a controller based on a neural network wont collide. But you can prove that a simple lidar based backup will not collide with a stationary object. Neural networks are great for complex decision process, but they cannot 100% gaurantee safety. Given the tesla pricepoint, and the falling price of lidars I don't see why one cannot be installed. Personally, I think it should be mandatory in all self driving cars to have a provable failsafe.
  • jayd16 6y ago
    The Model 3 has a front facing radar.
• ronreiter 6y ago
  This literally is the you-had-one-job of self driving cars
• lvs 6y ago
  The report that air bags failed to deploy should be a huge red flag for NTSB. I hope they are sending a team to investigate. Airbag deployment failure should not happen in any modern vehicle.
• Causality1 6y ago
  Why is it auto-safety systems just can't handle objects in the road? That's all I personally need it for. I try not to drive in such as a way as to be vulnerable to bad behavior from other cars but it'd sure be nice to not plow into nearly-invisible semi-truck tires laying in the road at night.