5 ms·
(Tesla) has twice attempted to drive directly into a train while in FSD mode
- Animats 2y agoDiscussed in many places. This is the original video.
- foobarian 2y agoI don't get why people even consider driving hands-off on any of the autopilot-like technologies around. Maybe in the cleanest possible conditions on a highway in slow moving rush hour traffic; and even then hands on the wheel expecting the worst. Do folks really have that much faith in technology? :-O
- varelse 2y ago[dead]
- rwmj 2y agoIsn't it because it is advertised as "Full Self Driving"? Not everyone is an expert on AI or follows auto news in detail.
- sli 2y agoThe name "Autopilot" for their lane-keeping feature clearly relies on most people not knowing how autopilot works in planes, because it's heavily implied to be autonomous driving and it's not. It's just a Tesla scam.
- watermelon0 2y agoAFAIK pilots don't need to hold the yoke for autopilot to work on planes, but this is required in Tesla.
- chucksta 2y agoI don't think its clear they are relying on people not knowing how autopilot works on planes because I don't think most people know autopilot on planes exist for the most part. 10-15% of the US population has never been on a plane.
- bell-cot 2y agoTHIS. If you asked a bunch of pilots about the capability level they'd expect from something called "autopilot", then compared with the answers you'd get from a bunch of Joe Averages...yeah. When people don't know squat, and they hear some Marketing-speak, and they'd like to believe - the next thing you know, they've convinced themselves that Santa's magic elves are making it all work, and nothing bad could ever possibly happen.
- konfusinomicon 2y agoas far as I'm concerned and I know it's not entirely feasible, but it should be all cars on the road are autopilot or none are. and until then if someone is driving in autopilot there should be a big ass flashing indicator light that I can see to warn me that extra caution needs to be taken around that car
- nradov 2y agoThe Mercedes-Benz Drive Pilot system has special warning lights when operating in autonomous mode. It is generally considered to be technically superior to Tesla's system in most respects. https://group.mercedes-benz.com/innovation/product-innovation/autonomous-driving/drive-pilot-marker-lights.html https://group.mercedes-benz.com/innovation/product-innovatio...
- konfusinomicon 2y agodefinitely a step in the right direction but needs more flashyness
- rocqua 2y ago99% reliable is very convincing at looking like 100% reliable. People get comfortable with things pretty quickly.
- samegene321 2y agoYes and every halving of that 1% error is more than twice as hard to do with neural net training. At some point more principled control algorithms are needed.
- tiahura 2y agoBecause for all the angst and outrage at HN, ask yourself how many people do you personally know that have been killed by Autopilot?
- PhasmaFelis 2y ago"It hasn't killed any one I know" is a deeply inadequate standard. There's a lot of monstrously deadly things that haven't killed anyone I know.
- rsynnott 2y agoI mean I don’t know anyone who’s been killed by a THERAC-25, either, but I wouldn’t be in a hurry to go under one.
- reliablereason 2y agoI would not trust something that drives as fast as that car does in that type of fog. But I trust my open pilot device, but a large part of that is that I can see on the screen actually what it intends to do.
- kelnos 2y agoI've been pretty happy with the ADAS system on my Mercedes E-Class. Over time I've learned what it does well and what it doesn't do well. But yes, of course, hands on the wheel 100% of the time, same amount of attention as if I were driving fully manually. I see it as mainly a way to avoid leg/foot fatigue, especially in heavier traffic, and also as a sort of backup. I think of myself still as the "primary system", but the ADAS will help me stay in lane if I get distracted on a long drive, or if I don't notice the car in front of my braking hard. I haven't needed it to save me from anything (yet, knock on wood), but it's nice to have it there. It is absolutely bonkers to me that some people think these sorts of systems allow them to not pay attention, take their hands off the wheel, do things on their phones, etc. Even Tesla's so-called "FSD" (with the scariest of scare quotes) isn't an excuse for that sort of thing.
- beders 2y agoI'll use the autopilot in regular traffic on highways only. But I'm also aware that glare will mess with the cameras in my Mach-E. Also, the driver attention system warns me if I'm not paying attention to traffic. I.e. know when to use it and be aware of its limitations.
- jh00ker 2y agoI use it to take over while I take my eyes off the road for 1-3 seconds to adjust the GPS navigation / radio, or plug my phone in for charging when it's otherwise safe-ish.
- nebula8804 2y agoI have been doing an experiment for a while now where I rent different EVs for a day from Turo to see if there is one special one that will make me give up my favorite car that I have had for years (06 Mazda 3) Rented a Tesla for an entire day yesterday and I see the appeal. I only tried autopilot on the highway with straight or curvy lanes. Handled it perfectly and really helped cover like 70% of my road trip (the part with straight roads for miles) Before this, I rented a Ford mach-e about a month ago with the latest Bluecruise and did the same test, was constantly "ping ponging" back and forth in the lane and essentially gave me no confidence whatsoever in its abilities. I then tested a Bolt EUV after the Ford but before the Tesla. Unfortunately I wasn't able to get a Bolt EUV with Supercruise but at this point it does not matter, the supercruise is on the Bolt EUV which is not comparable to the Tesla in other areas and all the other GM models which have it blow past my budget. Autopilot is really a relaxing system when you have been driving for hours. It politely nags you for what seems like every 5 mins so its not like I can fall asleep (at least I wouldn't dare do that) but it really takes some cognitive load off. Was also raining heavily for some of the day and Autopilot detected this and made it clear on the screen to be extra cautious. Seemed to handle it perfectly fine but I was overly cautious anyway. My assessment at the end of the day is that given Tesla's excellent software compared to Ford and Chevy, their amazing charging experience compared to EVgo/Chargepoint, and the overall Autopilot system, their cars are in a league of their own compared to other EVs and its just a waste of time to even consider the others. I am still not sold on EVs vs getting another Gas car but Tesla jumped to the top of my list for an EV. Like I mentioned I decided im likely to not even waste my time and money trying VW and Hyundai (they were next on my list to test). Not sure yet though. Wish we could get at least the basic highway version on all cars. I think it would be a big positive for so many people. I never really believed until I finally tried it out for an entire day. People NEED to give it a day of actual testing before making an assessment.
- nateguchi 2y agoSurprised it allows FSD in such poor visibility conditions. Surely they need to do some work on a reported visibility confidence level where FSD disables if it is not confident in the amount of visibility it has.
- mirsadm 2y agoIf it knew that it would also be able to drive by itself.
- resolutebat 2y agoAutosteer (lane following) will disengage if it can't see lane markings. I'm genuinely surprised FSD was active in these weather conditions (and as other posters have suggested, it may not be FSD at all).
- timeon 2y agoThe problem AIs have is that they are often confidentially incorrect.
- furyofantares 2y agoThat sounds definitely untrue.
- thorum 2y agoI don’t understand why self-driving systems don’t have any kind of fallback protections. Like however hard full self driving is to get right, surely a hard rule for “don’t drive at full speed into solid objects, no matter how good an idea it seems at the time to the AI” is not that hard to implement?
- dheera 2y agoThat video also seems like they were driving insanely fast on a foggy day. Quite possibly setting the speed offset way higher than the speed limit. A human could have easily not seen the train as well at that speed. During fog your speed limit should be determined by your braking distance and visibility. Period. I wonder whether the autopilot would have performed better if travelling at a more reasonable speed.
- robryan 2y agoYou can see the flashing light from way back. You would also probably hear the train.
- GuB-42 2y agoHe doesn't seem to drive that fast, he drove at a constant speed only to stop at the very last second. Had he been driving too fast, he would have been right into the train and probably not here to post on Twitter. The car didn't slow down even as the train was clearly visible. An attentive human would have not only seen the train but also the very obvious flashing lights and have absolutely not problem stopping in time. EDIT: From the link, it seems FSD was set by the car to about 61~63 MPH on a 55 MPH road, which is the speed most people drive on ( https://teslamotorsclub.com/tmc/threads/ap-fsd-related-crashes.264798/page-9#post-8264597 https://teslamotorsclub.com/tmc/threads/ap-fsd-related-crash... ). Considering the conditions, it is a little fast, but not unreasonably so. It took about 5 seconds from seeing the train to the crash, most of it at constant speed, so, a distance of about 140m. At that speed, stopping distance is about 100m, enough to stop in time. And that's for the unlit train, lights can be seen from much further as they are all visible at the start of the video.
- 2y ago
- Someone1234 2y agoI predict Tesla will claim that FSD disconnected 5-seconds before impact, therefore it is the driver's fault entirely. Or some vague thing about FSD giving an attention warning within the 15-minutes before impact.
- x86x87 2y agoFSD? More like Full Self Blaming!
- synarchefriend 2y agoThis happened once before and the driver still wasn't paying attention at as obvious a hazard as a train crossing? Not sure I trust them to distinguish between autopilot and FSD. Although of course FSD is not actually capable of driving without supervision so it could easily be true.
- bpodgursky 2y agoA lot of these "FSD" failures end up being Autopilot w/ lanekeeping.
- deleted 2y ago[deleted]
- phamilton4 2y agoPerhaps it's time they consider lidar? Elon hasn't seen the WHY yet? Tesla CEO Elon Musk has stated that cameras are the only sensors self-driving vehicles need, and that Tesla's electric vehicles do not use lidars. Musk has previously called lidar "expensive appendices" and "a fools' errand". He has also said that any company that relies on lidar for its autonomous capabilities is "doomed".
- renegade-otter 2y agoNow, try to be an engineer that tries to "correct" him. Musk's ego is so fat, if it walks by in front of your TV, you will miss the whole season.
- mvdtnz 2y agoInteresting how anything related to Tesla never lasts more than a few minutes on the front page.
- denysvitali 2y agoTesla is definitely at fault here, but in my experience, when camera visibility is very poor (only a few meters ahead), the driver's visibility is even worse. I've been in situations with heavy rain where I couldn't see more than a few meters ahead, yet the front camera's recording showed more than what I could see with my own eyes.
- neilv 2y ago> "I have owned my Tesla for less than a year, and within the last six months, it has twice attempted to drive directly into a passing train while in FSD mode. After the first time, I would hope the victim of this reckless product&marketing would go to a lawyer. (Maybe in a cab.)
- okintheory 2y agoWhen Ken Klippenstein obtained footage of an FSD Tesla causing a major accident [1] and wrote about it, he got blocked from being found through X/Twitter search [2]. I remember verifying the search block on my phone at the time. If I were a US lawmaker, I'd take a moment to reflect on whether a company such as X/Twitter, which actively censors journalists to protect the financial interests of its owners, should really be granted Section 230 protection. [1] https://x.com/kenklippenstein/status/1612848872061128704 https://x.com/kenklippenstein/status/1612848872061128704 [2] https://x.com/kenklippenstein/status/1604574747945275396 https://x.com/kenklippenstein/status/1604574747945275396
- tzs 2y agoThe whole point of section 230 is to let sites moderate user generated content without becoming responsible for what is in that content. What Twitter is doing falls squarely within the letter and the spirit of section 230.
- okintheory 2y agoI'm not arguing that X/Twitter is doing something that prevents it from being protected by section 230 as it exists. But, when Congress granted these protections, I think they had in mind things like forum moderation to remove hate speech, not selectively censoring journalists that write critically of a company. So one could rethink whether companies that censor journalists should be protected.
- paul7986 2y agolol you trust Elon Musk and his PT Barnum salesmanship with your life and really any self driving technology ... recently seen two waymo's driving on the wrong side of the road in San Fran (https://www.reddit.com/r/waymo/comments/1c9oefx/waymo_going_down_the_street_the_wrong_way/ https://www.reddit.com/r/waymo/comments/1c9oefx/waymo_going_...) and in Phoenix (https://www.12news.com/article/traffic/wrong-way-waymo-arizona/75-3036bffe-4824-4056-bfeb-d1377fd0e4a6 https://www.12news.com/article/traffic/wrong-way-waymo-arizo...) Im not against the technology it's just in it's infancy and those using it are paying tons of money to be guinea pigs so progress can be made.
- cladopa 2y agoTotally crazy that someone drives that fast with fog like this, knowing that there is a rail crossing there. No matter what kind of autopilot I have, I will be always careful around those places. It also does not make any sense that Tesla does not mark in the map rail crossings as places of extreme danger that make the vehicle go slower automatically. Rail crossings, like road crossings are death traps. They should not exist, they are bugs to be solved with roundabouts and bridges or underground pass. Also does not make sense that Tesla could not detect extremely bright lights indicating danger.
- dheera 2y agoI wouldn't be surprised if their working culture got rid of their most talented people. It is literally a burnout factory. That can easily be the source of all downstream problems.
- konfusinomicon 2y agoall the recent discussions and controversy around auto pilot and it made me wonder if there is anything to be learned from the OG self driving car, the Amish horse and buggy. I grew up in a community where the Amish were quite prevalent, and there is an old and popular saying when an accident involving a car and a buggy happened that "it was the horses fault". not sure of how often horses walk into passing trains but I'd assume it's not very frequently
- rsynnott 2y agoThe speed is an advantage there from a safety point of view. You can get away with a lot with slow-moving vehicles.
- joshspankit 2y agoI’ve been reflecting on Musk’s “roads are designed for two eyes” quote. I’m not sure on the exact wording but he used it to justify dropping lidar and focusing on front-facing cameras. What got missed, I think, is that there are many cases where having only two eyes fails drivers. The most dangerous driving most of us will ever do is because of some quirk of the landscape, or some long-forgotten legal battle over where the road should be, or because people are cramming more and more in to very limited space. An excellent example that comes to mind is a steep hill leading up to a dense downtown. At the very top of the hill is an intersection with a two-way stop sign. You can’t see to the left because of the edge of a building, and the hill is steep enough that you have to kind of stomp on the gas to get up it after you stop. “Two eyes” utterly fails us here. If there were cameras facing left and right on each corner of the vehicle the problem would be solve as you could watch the oncoming lanes while stopped. Here too, two eyes fails us. You have to already know about the train crossing, or hear it, or intuit it’s presence some other way because once you see it it’s so late that you need to slam the brakes to avoid hitting it.
- Symbiote 2y agoThe red lights off the crossing are very clear, even in the fog. In any case, one should never drive so fast they can't stop within the visible distance.
- calyth2018 2y agoFool me once, shame on you. Fool me twice, shame on me. Cause maybe the third time, someone's going to be a railroad statistic.