14 ms·
Tesla Autopilot Almost Steers Model 3 into Oncoming Train
- bananapub 4y agoit's absolutely surreal to me that 1) this is allowed to be used on public roads and 2) that Tesla is permitted to call whatever they've invented "self driving" or "autonomous".
- NelsonMinar 4y agoI really don't understand how Tesla markets something as "Autopilot" and "Full Self-Driving" and then also says "oh it's beta" and "you must be ready to take control of the car at any second".
- thathndude 4y agoI’m a lawyer, and I don’t get it. How hasn’t the FCC shut them down? I get it, they have an asterisk, and the asterisk makes clear that it’s not full self driving. But this feels to me like selling a product as boneless, including an asterisk, and then saying “actually not boneless, chew carefully”.
- D13Fd 4y ago> How hasn’t the FCC shut them down? Should be FTC, but yeah. This seems like a clear case of an unfair and deceptive trade practice that (literally, physically) injures consumers.
- deleted 4y ago[deleted]
- paulcole 4y agoIt’s not autopilot, it’s Autopilot. They’re making up their own branded feature called Autopilot which is nothing like what any reasonable person would consider to be autopilot. If you go back through the years here, you’ll see increasing care they use with their language: https://web.archive.org/web/*/Tesla.com/autopilot https://web.archive.org/web/*/Tesla.com/autopilot Look at the snake oil they were peddling in 2016: > All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver. Not only was this clearly false in 2016, it’s clearly false today, nearly 6 years later.
- Someone 4y agoFrom the viewpoint of Tesla’s marketing and legal departments, the nice thing about that phrase is that there’s no way to prove it clearly is false. Their claim isn’t about their cars; it’s about the fact that some software can be created that gives their cars “a self-driving capability at a safety level substantially greater than that of a human driver”. Even if they never manage to deliver such software, that’s no proof such software doesn’t exist.
- ncmncm 4y ago??? I thought that was what it was for. Or maybe for making wacky collision vids on YouTube? Tesla just closed their whole self-driving office in San Mateo, and fired ("for poor performance") most of the staff, directing some to come to work across the bay. For now. People still employed on that activity would best update their résumés. People who paid for self-driving already? I would ask for a refund, in your shoes. (I wonder if they will comply.) People fired for cause without due process? Sue. Really. How scummy is it to do a mass layoff, and make toxic shit up just to avoid complying with laws for mass layoffs? Supposedly the "important" people are still employed. For now.
- henrikschroder 4y ago> Supposedly the "important" people are still employed. Karpathy is still on a "sabbatical". The odds of him returning from that is pretty much zero.
- whiplash451 4y agoWait. Karpathy is on sabbatical? Says who?
- enragedcacti 4y ago"Btw, @karpathy is on a ~4 month sabbatical." https://twitter.com/elonmusk/status/1508110810614181890 https://twitter.com/elonmusk/status/1508110810614181890
- fdhsbjasvh 4y agoSays Karparthy IRL. And yes I doubt he’ll return.
- fdhsbjasvh 4y agoIt's official now; https://twitter.com/karpathy/status/1547332300186066944 https://twitter.com/karpathy/status/1547332300186066944
- darkerside 4y agoWas the train running a red light? This seems confusing for a human driver, too. I'm usually looking for cars and pedestrians, not trains when turning left.
- mathieuh 4y agoIt’s a tram, not a train, and many cities have trams as part of their public transport system
- MonkeyMalarky 4y agoThough many cities have them they're sort of glossed over in drivers ed and it clearly shows at times.
- zelos 4y agoDo people really need a lesson to know not to turn just in front of a tram?
- mrlonglong 4y agoPeople still do despite knowing this. Loss of licence is the only cure. My biggest peeve are idiots pulling in too close in front of a 44 ton truck just so they can turn left.
- soco 4y agoTo many the loss of own car is a good enough cure. Trams can rarely pull a full stop in that time.
- jacquesm 4y agoIf they do it may end up being a very permanent lesson, those things may look flimsy but they definitely aren't. A frontal collision between a truck and a tram in Amsterdam left the tram without so much as a dent, though it did have a couple of scratches in the paint work. The truck could have served as a mold to cast trams from.
- jaxn 4y agoI used it every day and it does almost all of my daily driving. If it never improves from where it is now, it will be worth the $6k I paid for it. And yeah, you gotta watch it and pay attention.
- rainsford 4y agoNo snark intended, but doesn't having to constantly pay attention and be ready to take over control negate a lot of the benefit of the self-driving features? It seems like you'd have to apply just as much mental energy to the task as if you were actually driving, so the only real benefit is that you don't have to physically make the motions. In fact in some ways it seems mentally worse, since when you're driving you aren't surprised by sudden turns in front of trams and the like. With self-driving, there's a constant possibility of those kind of surprises, so I would think it would feel a lot like riding in the passenger seat with a new teenage driver.
- jetbalsa 4y agoI used a Comma 3 (somewhat close to FSD, just simpler) and I found that it allows to you look forward in time more then you normally would. Normally you might be looking at the first 50ft for issues, now I find I'm able to expand out to 200ft and look around more for traffic.
- jacquesm 4y agoNot having that much situational awareness sounds super dangerous. 50 ft is way too little for safe driving.
- coryrc 4y agoEvery day I drive, I see dozens of incidents where I think someone should be ticketed for reckless driving: tailgating, excessive speed, unexpected lane changes, etc They never, ever are. We do not enforce good driving except after people die (or even then, vehicular homicide has low sentences).
- rvz 4y agoThis contraption is literally a disaster on wheels. Malfunctions on the public roads and essentially puts the driver and others on the road at risk. It's other counterpart FSD, 'Fools Self Driving' is even worse and is already being investigated. There is zero point in defending the incompleteness of this so-called 'safety-critical' contraption. It simply does not work and is simply not ready.
- sva_ 4y agoIt's a bit sad to imagine that Tesla's - perhaps to be considered reckless - release of their alpha/beta software on public roads could lead to legislation making it a lot harder for future companies trying to develop self-driving cars. So maybe Musk's pioneering work in this will actually set the field back in some way.
- paywallasinbeer 4y agohttps://youtu.be/yxX4tDkSc_g?t=978 https://youtu.be/yxX4tDkSc_g?t=978 is the linked video and it happens at 16:20.
- belter 4y agoSame video, another almost disaster forcing driver to break hard: https://youtu.be/yxX4tDkSc_g?t=428 https://youtu.be/yxX4tDkSc_g?t=428 It's a tram not a train, the driver does call it a tram, but labels the video a train...It's the new era of word twisting..inspired by FSD... Edit: Still the same video, here driver has to break hard. Would run a red light, get several people killed at that large, busy, and fast intersection: https://youtu.be/yxX4tDkSc_g?t=738 https://youtu.be/yxX4tDkSc_g?t=738 Insane....just insane, and that is being diplomatic.
- juanani 4y ago
- deleted 4y ago[deleted]
- saos 4y agoTheres really no need to for autonomous driving is there>
- throwaway5959 4y agoWhy is the US allowing this beta testing of a feature that impacts safety to continue? There have to be at least dozens of documented cases of this being unsafe. Have regulators just given up or are they that incompetent?
- chrisseaton 4y ago> There have to be at least dozens of documented cases of this being unsafe. The human in the loop seems to catch the mistake just fine? Doesn't seem to be even remotely close to an accident?
- skywhopper 4y agoDid you miss the ones where the self-driving Teslas plowed into fire trucks on the side of the road?
- kadoban 4y agoAny system based on a human doing nothing for long periods and then being expected to act swiftly and accurately is _deeply_ flawed. Human attention doesn't work like that. This is russian roulette with the general public forced to play.
- mwint 4y agoSo FSD beta testing has been around for what, a couple years? To continue the tortured analogy: We’ve been playing this Russian roulette for millions of miles. How many times to people have to be “almost” shot, but not actually shot, before we concede that maybe the gun has no bullets?
- kadoban 4y ago> How many times to people have to be “almost” shot, but not actually shot, before we concede that maybe the gun has no bullets? You say that like we have anything like good data. We have pretty much what Tesla's marketing team put together on their website, and _maybe_ what some cities or states have cobbled together. And before we assume that Telsa wouldn't lie, let's remember that they call this "Full Self-Driving".
- mountainriver 4y agoThe fact that this can be solved using LiDAR, just seems irresponsible to not
- makeworld 4y agoOh man, they're not using lidar? I had no idea, that's crazy they're just putting cars on the road just cameras and AI. Jeez.
- charles_kaw 4y agothey even removed front facing radar a few years back, to save on cost
- thedevelopnik 4y agoElon thinks that because humans can do all this with two eyes his cars should be able to do it with just cameras. Literal truth.
- ohgodplsno 4y agoI mean, what are eyes except constantly refocusing cameras with changeable fields of view, light intake, quick focus backed by a brain that can take action on what is seen even before you actually visualise it? To Musk and his goblins, that's basically just cameras. But it's okay, people keep assuring me he's a great engineer. Or a complete fucking sociopath that's lied his way to success but hey
- deleted 4y ago[deleted]
- AlexandrB 4y agoSeems like another example of Moravec's paradox[1]. The higher order decisions involved in driving seem hard because we're conscious of them. But our visual system is assumed to be "just two cameras" because we have no conscious awareness of what's going on when we create an internal representation of our surroundings using our eyes. [1] https://en.m.wikipedia.org/wiki/Moravec%27s_paradox https://en.m.wikipedia.org/wiki/Moravec%27s_paradox
- 988747 4y agoOne thing about this video that caught my attention: around 14:40, Tesla is in the "forward" lane, but it tries to make a left turn. While I agree that this is against the rules, my experience with driving around a big city is: half of the human drivers would try to do the same, if they accidentally ended up in the wrong lane. So, if Tesla is using data captured while human is driving, can it pick up bad habits as well ?
- TheChaplain 4y agoYes, it would not surprise me. I once had Google Maps asking me to go against a one-way road, but I think their algorithms are better as it is very rare I encounter stupidities like that.
- AlexandrB 4y agoI think more alarming was at 12:20 when it tried to go through an intersection on a solid red.
- salawat 4y agoGarbage in, garbage out. The penultimate product of Machine Learning... Is the oddest human savant to ever exist.
- deng 4y agoHere's the thing: I don't think the Tesla even realizes it is in the wrong lane.
- mancerayder 4y agoThe comments are a pile-on of criticism and how can this be legal. The question is what percentage of self driving hours lead to a situation like this, is it high enough to determine it's unsafe compared to non autopilot or statistically an outlier?
- bamboozled 4y agoIf I understand you correctly, you're arguing that having machines making mistakes maybe ok if it's statistically less likely to happen than humans?
- deleted 4y ago[deleted]
- mancerayder 4y agoI'm asking the question of how common it is. If I understand you correctly, self-driving should be banned if there's 1 error?
- UncleEntity 4y agoThere’s a built in autoban feature, one head on collision with a train activates it. Unfortunately it sucks for all the innocent people on the train.
- AshamedCaptain 4y agoWe actually do that with many industries where such errors result in loss of human life. Entire plane fleets being grounded and things like that.
- ncmncm 4y agoThey waited until two crashed before they grounded 737MAX. Two too many. They already knew.
- BurningFrog 4y ago
- deleted 4y ago[deleted]
- glintik 4y agoSome offtop, but - I’m waiting for the moment when I can choose autopilot profile: F1 racer, Mr. President’s driver, Pensioner etc. Will be funny.
- rsynnott 4y agoThis feels like an escalation from the usual cars and bollards and things. Possibly The Algorithm has become bored, and we can expect a series of robocars menacing more and more unlikely vehicles, culminating in a spectacular near-miss with a zeppelin.
- LightG 4y agoHahahha! I laughed. Not worried about the downvotes.
- salawat 4y agoA Tesla on Autopilot took out a helicopter today... Elon Musk commented "That was totally rad, and his engineers were in the process of attempting to replicate the finding as a new "Vigilante" mode to become available next Thursday, probably."
- SV_BubbleTime 4y agoI’m not going to be mad at the CEO with a sense of humor. No one was hurt, seems like they’ll pay for it. Pretty rare scenario. I feel like we’ve lost a lot of our will to fail, and also joke about it.
- vachina 4y agoI like how the tram and car models twitch in the 3D view. Very confidence inducing.
- mensetmanusman 4y agoIn context there are 17k crashes per day in the us. 5800 annually involve a car and a train.
- jcundososna 4y agoNot sure what this is supposed to tell me. The number of cars driven by humans and vehicles driven on autopilot is different by a huge magnitude, so whats the point of comparing absolute numbers of crashes. At the very least to make a fair comparison we need to use crashes per cars driven in a day by humans and compare that with autonomous vehicles. The fact that these bugs actually can be fatal, some even random people who have nothing to do with this test should be very concerning at the very least and not bruahed away as but x accidents happen anyway already.
- js2 4y agoUnprotected left turns are one of the most dangerous situations in driving. I consider myself a competent driver, haven't been in an accident in over two decades, and I barely trust myself to make an unprotected left. I've heard that UPS (used to?) ban their drivers from making unprotected lefts. Not in a million years would I ever trust this maneuver to software. Humans screw it up all the time: https://old.reddit.com/r/IdiotsInCars/comments/uks7c2/unprotected_left_turns_are_hard_for_some_people/ https://old.reddit.com/r/IdiotsInCars/comments/uks7c2/unprot... (No accident, the driver just gets stuck when the light changes.) Here's a recent video of a Tesla driver videoing his car making the same unprotected left multiple times: https://www.youtube.com/watch?v=j0Kvsq9xB8c https://www.youtube.com/watch?v=j0Kvsq9xB8c Given how much supervision it requires, I don't see the advantage of letting the car do the driving here.
- aidenn0 4y agoIn theory, an unprotected left should be easier for software than a human. The issue with them for humans is that there are too many places traffic (and pedestrians) can come from. For computers, once you solve the problem of seeing traffic coming from one direction, Nx the computing power can see traffic coming from N directions. The problem Tesla has is that they still can't do basic things reliably enough
- js2 4y agoHuman drivers are also impatient, distractible, poor at judging speed, and have visual blind spots. But a big problem with a lot of unprotected lefts is poor visibility of oncoming traffic. The car just isn't in a great position to see down the road. Tesla has to deal with that same limitation.
- salawat 4y agoTurns out computers have the same problems without the right hardware and programming, much of which would be dependent on the public to implement. There's a reason these problems stick around.
- 4y ago
- petilon 4y agoMeanwhile Cruise just obtained permission to charge passengers for driverless rides in San Francisco [1]. Cruise has been cleared to charge for rides in vehicles that will have no other people in them besides the passengers. There will be no back-up human driver present to take control if something goes wrong. Why is Tesla so far behind? Tesla is generating these autopilot mishap news on such a regular basis that it is giving the entire industry a bad name. [1] https://www.npr.org/2022/06/03/1102922330/driverless-self-driving-taxis-san-francisco-gm-cruise https://www.npr.org/2022/06/03/1102922330/driverless-self-dr...
- tenuousemphasis 4y agoThe answer is that Tesla is creating a general solution, whereas Waymo and Cruise and targeting specific geographical areas which they can map precisely.
- petilon 4y agoWaymo and Cruise are also using more expensive, bulky and unsightly hardware, including lidar. They are starting at the top, with a solution that works, then working their way down. Their approach is confidence-inspiring. Tesla is starting at the bottom. The frequent mishap stories coming from Tesla does not inspire confidence.
- dmitriid 4y agoYou can also start at the bottom like most European manufacturers, and slowly build up to autonomous driving. That is, start with collision warning, lane assist etc. Boom, you have a Volvo that drives itself on a highway, and requires driver assistance elsewhere.
- moneywoes 4y agoDoesn’t most taxis revenue come from city centers only ( 80-20 rule) seems as if Cruise’s solution would be prudent
- belter 4y ago
- deng 4y agoActually, that steering into the oncoming train is just one of several severe instances in that video: * Almost driving right into a barrier because road is closed: https://youtu.be/yxX4tDkSc_g?t=426 https://youtu.be/yxX4tDkSc_g?t=426 * Choosing the wrong lane and getting confused: https://youtu.be/yxX4tDkSc_g?t=666 https://youtu.be/yxX4tDkSc_g?t=666 * Trying to run a red light, right into traffic: https://youtu.be/yxX4tDkSc_g?t=732 https://youtu.be/yxX4tDkSc_g?t=732 * Stopping in the middle of an intersection without reason: https://youtu.be/yxX4tDkSc_g?t=789 https://youtu.be/yxX4tDkSc_g?t=789 * Choosing the wrong lane for a left turn: https://youtu.be/yxX4tDkSc_g?t=805 https://youtu.be/yxX4tDkSc_g?t=805 * Constantly activating and deactivating the left turn-signal without reason (you can't even turn left there): https://youtu.be/yxX4tDkSc_g?t=902 https://youtu.be/yxX4tDkSc_g?t=902 * Unable to make a proper left turn, almost running into pedestrians: https://youtu.be/yxX4tDkSc_g?t=1033 https://youtu.be/yxX4tDkSc_g?t=1033 All of this during maybe 30min of driving. Also, these are just the most severe instances, the Tesla just drives very awkwardly. It is changing lanes way too often (also on intersections) without rhyme or reason, and it weirdly creeps forward during red lights, getting very close to crossing traffic, for no reason at all. All in all, this is an incredibly poor performance.
- spaceman_2020 4y agoI just can’t understand why you’d give up control to a system that’s clearly still in beta. The cost of failure is way too high to take the risk.
- cloutchaser 4y agoThe whole point should be you don't. You are giving live feedback to the most advanced ML learning system ever (probably). Which hopefully in a few years will actually work. You are not (currently) using an autonomous car. If you think so you are an idiot.
- yladiz 4y ago> You are not (currently) using an autonomous car. If you think so you are an idiot. Let's define autonomous as self driving. According to their marketing page[1]: > Tesla cars come standard with advanced hardware capable of providing Autopilot features, and full self-driving capabilities—through software updates designed to improve functionality over time. So it has capabilities of autonomous driving, which means a Tesla can be considered an autonomous car. Unless I'm an idiot for trusting what their marketing page tells me, of course. 1: https://www.tesla.com/autopilot https://www.tesla.com/autopilot
- ra7 4y agoDang — can we replace ‘Autopilot’ with ‘FSD Beta’ in the title? It’s an important distinction as they do different things.
- ra7 4y agoHere is another idiot "testing" it out in front a school bus. The car wouldn't stop so he had to intervene: https://youtu.be/zQO6RJCUPEI?t=10 https://youtu.be/zQO6RJCUPEI?t=10 I'm continually surprised how it's still being allowed on public roads. Tesla claims it's not an autonomous system to regulators and refuses to provide any data, while marketing it as imminent Level 5 self driving. It seems like it will take someone to be seriously injured or die for any action to occur.
- belter 4y agoNo it won't. The US is the land of regulatory capture. "A Tesla driver is charged in a crash involving Autopilot that killed 2 people": https://www.npr.org/2022/01/18/1073857310/tesla-autopilot-crash-charges https://www.npr.org/2022/01/18/1073857310/tesla-autopilot-cr...
- deleted 4y ago[deleted]
- aiisjustanif 4y agoWhy is the guy an idiot?
- synu 4y agoI don't know if the guy is an idiot or not but I personally wouldn't have handed the wheel over to an AI right next to a school bus in order to find out if it was going to run the kids over.
- curiousgal 4y agoI'm just sitting here vexed about the mislabeling of a tram as a train. Like the guy himself called it a tram but then used the word train in the title. If you have to lie to get appint across it just takes away from that point, valid as it may be.
- nkotov 4y agoBasic autopilot for highway driving has been amazing but I turn it off the moment I get on to a street because I'm not comfortable with it. I've had both a Model S and now have a Model 3. I skipped the FSD package on the model 3 because it's just not ready.
- karmicthreat 4y agoI'm a terrible driver and its probably best there is an automated system to back me up. And from my experience with my daily commute, there are also many other terrible drivers on the road. While Autopilot isn't perfect, its better than just throwing your hands up and saying nothing can be done about accidents.
- fortran77 4y agoThis is model 3. You have to expect some compromises on the budget systems. Do the flagship models (S, Plaid) have better systems?
- MrThoughtful 4y agoIs anybody publicly tracking Teslas self-driving progress over time? I have been googling a bit, but not found anything: Is somebody out there driving the same route again and again , tracking the number of disengagements over time? There are gazillions of YouTubers who constantly put out videos how they drive their Teslas. But they all do the opposite: They try to think about new routes for every video. I wonder if there is not a single one out there who tracks and documents the progress?
- CommieBobDole 4y agoI always post this in these threads, but my one experience driving a friend's Tesla on autopilot involved, after about two or three minutes of driving, a sudden attempt at 'correcting' onto the wrong side of a divided road at an intersection. It was admittedly sort of a weird intersection (oddly-angled 4-way on a curve) but it did feel awfully enthusiastic about driving into oncoming traffic.
- LightG 4y agoWouldn't touch anything Tesla AI'd with your bargepole ... it's a failed project.
- Gatsky 4y agoWe are living in the ‘good old days’ when you could deploy your self driving car with minimal regulatory oversight. In 5 years time it will be very different. The company that collects the most quality data in this laissez-faire environment may gain an unassailable lead.
- blinding-streak 4y agoIs this even news any more? Autopilot is nowhere near ready for prime time. There's overwhelming evidence for it.
- redsid 4y agoPersonal anecdote. I was behind a Tesla on a bit of mountainous stretch (single lane) on my drive home on a Sunday eve - so literally no traffic, and noticed the Tesla was painfully slow and almost at standstill. I realized it was on autopilot and literally sticking to the speed limits although the driver could have easily done at least 10mph over. The big truth that dawned on me was, here in northeast US with mostly single lanes, these level X driving cars are going to stick to speed limits, and am totally curious to see how road rage is going to show up
- verisimi 4y agoI actually think that dropping the speed limits is part of the strategy (in urban roads near speeds have been changed from 30 to 20) is to make the roads safer for self-driving cars. So, I think self-driving is going to be a slow experience. Maybe once everyone has been slowed down, they'll bring out a premium self-driving product, where is you are rich your car is allowed to move more quickly.
- hristov 4y agoI think teslas tend to automatically choose 5 mph above the speed limit when you turn on autopilot.
- cpuguy83 4y agoKey word here is "almost". No need to get bent out of shape, especially when the Tesla didn't. /s
- billfor 4y agoThe FSD beta signup clearly says autopilot may do the wrong thing at the worst time, so the user should expect that IF they want to try the FSD early access. Otherwise drive it manually or don’t opt in to FSD. Most of the time my Tesla tries to turn into traffic is when the GPS is not entirely fixed so the car thinks it is in a different position. Of course this video is in full screen mode so we can’t see the GPS info. I stay alert and enjoy testing this new and amazing software. I am responsible and if I’m not comfortable with the risk I can always turn it off or go back to the regular software.
- omega3 4y agoI'm not familiar with Tesla UI, is it possible to say from this video that it was indeed the autopilot that was making the turn?
- labrador 4y agoGoogle asks me to solve CAPTCHA's of crossing walks and traffic lights because robots can't do it. Tesla self-driving AI can safely determine crosswalks and traffic lights. Which of these two statements are true?
- gruez 4y ago> Google asks me to solve CAPTCHA's of crossing walks and traffic lights because robots can't do it. Any evidence this premise is true? It's entirely possible that they perfected crosswalk/traffic light identification, and they're continuing to use it for captcha purposes because it's easier than rewriting the captcha to use another form of images.
- labrador 4y agoWhy would they crowd source image identification that is already solved? There are plenty of other images they need human help on. Besides if robots can do it then someone would be using them to solve CAPTCHAs. I recently started using a VPN and now google asks me all the time to solve them before completing my search, so I'm assuming that robotically searching Google is something people want to do.
- gruez 4y ago> Why would they crowd source image identification that is already solved? There are plenty of other images they need human help on. Because they'll need to set up an entire pipeline to get those images into the system. Meanwhile pictures from streetview is plentiful and free (both in terms of licensing costs and development costs) >Besides if robots can do it then someone would be using them to solve CAPTCHAs. Maybe such models are in the hands of the top AI companies (ie. companies making autopilot systems), and they're not really interested in starting a side business of captcha bypass? The other possibility is that they're already in the hands of bad guys, they're just putting it under wraps so they can monetize it and not get it detected. For instance there's this: https://addons.mozilla.org/en-US/firefox/addon/buster-captcha-solver/ https://addons.mozilla.org/en-US/firefox/addon/buster-captch..., so the idea isn't too far fetched.