9 ms·
Tesla FSD Beta 10 veers into pedestrians with illegal right turn
- nodesocket 5y ago2021 Model Y Owner. Hard to tell, but it looks like that street is a one-way heading right which is legal. However, not yielding for pedestrians is no bueno though they were just barely off the sidewalk into the street when it turned.
- jdminhbg 5y agoIt's a one-way to the right but there's a no right turn sign on the pillar to the left of the street as you approach it. Not signed great, but still not a legal turn.
- benjaminjackman 5y agoThere is sign on the wire near the traffic light that says no turns, not sure if it only applies to the other lane though.
- gpm 5y agoThere's also a sign on this half of the traffic wire that looks like it might be saying "yes turns" (low resolution, but I can't imagine what the sign says with an arrow pointing right other then that)
- Alupis 5y agoThat sign says "One Way". Not "yes turns"... I don't think I've ever seen a sign saying that, actually, it's either "No Turns" or no sign at all, since by default turns are allowed.
- gpm 5y agoI've definitely seen signs that say "you can turn here" and "you must turn here", admittedly not with language but with arrows (the "you can turn here" signs around here looking like a forwards arrow with a curved arrow bending off). They're not in general use, only for weird situations (like "two turning lanes" or "yes really this really strange intersection does work like this"). Probably regional though.
- mikequinlan 5y ago>by default turns are allowed. It depends on where you live. In downtown Vancouver there were arrows at each intersection telling you what directions of travel were legal.
- nodesocket 5y agoAhhhhh I see the no right turn sign on the pillar. That is confusing as there are also one-way signs (with arrows pointing to the right) further along. Moral of the story, do you think most humans would do a better job at this intersection than the Tesla did?
- blankobj 5y agoThat sign in the pillar, if I was driving, I would assume applied to the other lane on the left side of the pillar. I think the intent is to make sure you don't turn left if you are in the right lane of the pillar side, and vice versa.
- rocketraman 5y agoThere is a no right turn sign on the post to the left of the vehicle just before it begins the turn.
- obilgic 5y agoMisleading/Exaggerated HN title. Not an illegal turn. No turn sign applies to the left lane, and it's a one way street
- bagels 5y agoAgree, it didn't hit anything, even though it might've without intervention. "into" is the misleading part.
- gpm 5y agoRight, and it didn't "veer" either, it performed a normal turn. The pedestrians had just started crossing, so it was in the wrong, but barely. It's the sort of manoeuvre that you see human drivers do all the time.
- awsthro00945 5y agoThe entire talking point of FSD is that it's supposed to be better than human drivers. This ain't it. >it didn't "veer" either, it performed a normal turn. The definition of veer is "to change direction". This did that. Stop arguing semantics. >The pedestrians had just started crossing I fail to see how this matters at all. Even if the pedestrians hadn't started crossing yet, and were still on the sidewalk, it would have been wrong for the Tesla to make the turn without yielding to the peds.
- gpm 5y ago> I fail to see how this matters at all. Contrast it to "running into the pedestrians"...
- void_mint 5y ago> The entire talking point of FSD is that it's supposed to be better than human drivers. This ain't it. the entire point of a beta is that it's supposed to be production grade software?
- 5y ago
- gallerdude 5y agoI don’t know how the UX works, but it seems like it de-activated itself when it got closer to the pedestrians? Or was that a manual override?
- bagels 5y agoIf you hit brakes or steer, it disengages.
- awsthro00945 5y agoThe video in the Tweet replies is even worse, IMO: https://twitter.com/TaylorOgan/status/1437743602579423233 https://twitter.com/TaylorOgan/status/1437743602579423233 The Tesla FSD would've caused a t-bone in the first two clips, and a head-on collision at decent speed in the third clip. Yikes.
- bagels 5y agoHow long can they get away with charging 10k for something that is trying to crash the car in to bits?
- sidibe 5y agoEverything about FSD is truly mindblowing. They sold the product but are not letting most users get the version that tries to be autonomous (and thank god for that). It won't be available to most of these people on the car they purchased it for, and can't be transferred to other cars. That has to end in a refund or class action law suit right? And now they have a subscription available? That also doesn't include these features? Who are the suckers paying monthly for this, especially after years of failed promises about this product (that's giving the benefit of the doubt, I think "lies" is more appropriate at this point).
- jiggawatts 5y agoCompanies seem to be able to get away with these kinds of shenanigans in some jurisdictions, but what blows my mind is that the Australian Competition and Consumer Commission (ACCC) hasn't yet come down on them like a ton of bricks.
- BugsJustFindMe 5y agoShitty sudden turn against the no-turn sign without using a turn signal aside, why was it turning right at all when the route showed it going straight for another several blocks and then turning left?
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- GravitasFailure 5y agoI've dealt with similar but less extreme behavior in lane keeping systems where it senses the right-side reference veering away so it "corrects" and follows, causing a veer to the right, so it might not have been trying to turn but instead drive next to the crosswalk at the same distance it was from the curb. Regardless of what happened, that's a really bad bug to have.
- BugsJustFindMe 5y agoInteresting. This could explain why it doesn't signal. But honestly even veering like that should elicit a turn signal to notify other drivers of the move.
- GravitasFailure 5y agoI'm not sure an extra alert would be helpful since it's either going to alert constantly on streets that twist and turn or it's going to think it's doing nothing wrong and not alert. Same for the turn signal.
- BugsJustFindMe 5y agoI think it should. Having a car in front of you means that part of the view of the road is obstructed. IMO the veering car has a moral duty to signal to the next car especially if the veering is unexpected. Now the state law on signaling says "No person shall turn a vehicle..." but I can't find what the definition for "turn" is, so there may be no legal obligation, but I do see a social obligation.
- Aliabid94 5y agoIf you have to anxiously hover your hands over the wheel constantly like the driver is in this video, is it really helpful even if it doesn't crash? I don't own a Tesla, curious how other drivers are when on Autopilot.
- notJim 5y agoThis is the city-driving beta that's only available to a small number of people. I have read that at this point, it is more stressful than driving yourself, but it is not yet at the point where anything more is expected.
- alkonaut 5y agoAssuming it stays in limited beta until safer than humans (statistically causing 1/10th as many accidents or some acceptable level like that) and until people actually feel measurably more relaxed when using it before it’s released to the public. Who then believes it will even be this decade?
- shepherdjerred 5y agoTest driving a Tesla Model 3 with autopilot last year was one of the most stressful experiences I’ve ever had. It’s not at all intuitive, and I had no idea what autopilot would/wouldn’t handle.
- aclatuts 5y agoIt might be worth it for Tesla to build a voice system to tell the driver what manoeuvres the car will be doing soon. Although implementing might be just as hard as self driving in the first place.
- Doxin 5y agoI feel like if they could manage to get the self driving system to make a plan and stick to it it'd be a better driver already. In so many of these videos you can see the car dithering between two or more options, with the whole car doing these jittery unsure steering motions. At least if it would pick a damn option it'd have a 50/50 chance of picking the right one.
- drooby 5y agoLevel 3+ requires AGI for unmapped non-highway roads. Change my mind.
- bagels 5y agoWhat is agi?
- gpm 5y agoAGI implies things like "motivations" and "feelings" not just understand of the world (i.e. beyond understanding other things "motivations" and "feelings"), it's a fundamentally different problem. Even if you think you need extremely good world understanding to make a self driving car, AGI is irrelevant (and would actually make it much scary, because an AGI is perfectly capable of deciding to intentionally kill you).
- singularity2001 5y agoMost insightful comment my week. Maybe we need a new catchphrase CAI comprehensive AI as an intermediate goal to AGI. But then: maybe a car needs to understand the motivations of pedestrians and human drivers in order to predict their movements. And we are almost back to AGI requirements
- camjohnson26 5y agoRemember this is the build that Elon Musk said was almost ready for public release and would “blow your mind”, https://twitter.com/elonmusk/status/1435967157662150675?s=20 https://twitter.com/elonmusk/status/1435967157662150675?s=20 But there have been plenty of failures, and remember that the only people who have the beta are employees who are presumably under an NDA, and devoted fans who have publicly said they don’t want to post the disengagements because critics “take them out of context” Other failures: https://twitter.com/28delayslater/status/1437866328950517768?s=20 https://twitter.com/28delayslater/status/1437866328950517768... https://twitter.com/BS__Exposed/status/1437367617468129280?s=20 https://twitter.com/BS__Exposed/status/1437367617468129280?s... https://twitter.com/TaylorOgan/status/1437385266046586890?s=20 https://twitter.com/TaylorOgan/status/1437385266046586890?s=... https://twitter.com/bagguley/status/1437328015789367297?s=20 https://twitter.com/bagguley/status/1437328015789367297?s=20 https://twitter.com/TaylorOgan/status/1437405124004089863?s=20 https://twitter.com/TaylorOgan/status/1437405124004089863?s=... https://twitter.com/BS__Exposed/status/1437043592560791552?s=20 https://twitter.com/BS__Exposed/status/1437043592560791552?s... https://twitter.com/bagguley/status/1437048381042475008?s=20 https://twitter.com/bagguley/status/1437048381042475008?s=20 https://twitter.com/BS__Exposed/status/1436806907663159299?s=20 https://twitter.com/BS__Exposed/status/1436806907663159299?s...
- bagels 5y agoLooks like we are pretty close to robotaxi.
- mensetmanusman 5y agoIt would be super interesting for Tesla to release context after these events, e.g. this is the 628,392,292th right turn this year.
- ogwh 5y agoWhy is everyone freaking out over a beta test with a driver at the wheel ready to take over immediately? It's a beta test, finding bugs is kind of the point. Or have we truly reached the point now where we expect "stable" software to be of the same calibre as beta software so nobody makes a distinction between the two?
- woofyman 5y agoBecause it’s a 5000 lb weapon and the pedestrians didn’t consent to the beta.
- thatswrong0 5y agoI see people making similar mistakes pretty frequently. Not trying to excuse beta testing this software with live pedestrians (like, why on earth did it not signal? What was it's planning software doing?), but humans make this mistake too
- ahahahahah 5y agoIn addition, and unlike every other serious company in the space, Tesla is using random untrained drivers for testing this crap.
- colechristensen 5y agoBecause a car is a life critical system and nobody should want an automated vehicle, even if controlled by a human, to "accidentally" try to run over some humans. I doubt the pilot's ability to stop all missteps before an injury. The car should be much safer before it goes on the road, regardless of test status. This was a near miss and should be taken extremely seriously by highway safety regulators. >Or have we truly reached the point now where we expect "stable" software to be of the same calibre as beta software so nobody makes a distinction between the two? I expect every vehicle on the road to meet a safety standard. I do not accept "beta software" tests on public roads if this is the kind of mistake they can make.
- dham 5y agoI've been using Open Pilot for the last 3 months (Rav4 Hybrid). I've really enjoyed it a lot on the highways. It also works really well in stop and go traffic. I'm starting to agree with George Hotz about making driving chill. Highway driving is chill. Stop and go traffic is chill. There's nothing chill about letting a car drive in these situations (pulling out intersections, right hand turn on red, congested streets with people everywhere) and I'm used to my car driving itself. There's nothing chill about your car having unlimited torque on the steering wheel where it can do something gnarly. Making your car do these maneuvers has the opposite effect, making you more stressed rather than taking the stress away. I'll stick to open pilot for now. I'd rather just drive in these situations. With Tesla FSD you already have to have your hand on the wheel, might as well put your foot on the pedals.
- ProAm 5y agoWhich version of Open Pilot?
- dham 5y ago0.8.8 in laneless mode. Comma 2. Decided not to return and get Comma 3 because 2 does everything I need it to do right now.
- xiphias2 5y agoI started to agree with George Hotz that a half autonomous car should try to drive like a human because that's what humans can superwise efficiently.
- md2020 5y agoAgreed. Just waiting on a good BEV that has full openpilot compatibility, maybe Hyundai Ioniq 5. Openpilot has a great community as well.
- ruph123 5y agoSo from Googling, Open Pilot is more a fancy HUD and camera system that assists the driver but leaves all control with him/her? Did not follow Hotz for a while and the last thing I remember was that he was also just working on another full self driving startup.
- improv32 5y agoThis is in Seattle, on 5th ave. The concrete pillars are the monorail track. The street confused me driving as a human sometimes. There are more videos of the Tesla software being confused on this particular street floating around
- loeg 5y agoThis street slightly confuses me, too, but I've never decided to suddenly and without signaling dive into the crosswalk when pedestrians were in it.
- GravitasFailure 5y agoI thought that street looked familiar. There was a twitter thread[0] discussed[1] about two months ago showing a failure along the same street. [0]https://twitter.com/giacaglia/status/1414605317841702914 https://twitter.com/giacaglia/status/1414605317841702914 [1]https://news.ycombinator.com/item?id=27811853 https://news.ycombinator.com/item?id=27811853
- modeless 5y agoI believe that it is not illegal to turn right there. Both streets are one way, and the "no right turn" sign is for the lanes to the left of the pillars. For context, these pillars in the middle of the road are handled poorly by Tesla FSD and this guy is repeatedly stress testing the system by forcing it to drive there, for YouTube videos. The latest FSD build is the first one that actually recognizes the pillars as obstacles; earlier builds would simply plow into them. What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked, so the path planner decided it had to turn right despite that not tracking the planned route. As far as the pedestrians, it looks to me like there was plenty of space to turn into the near lane without hitting them, and the pedestrians shouldn't have been starting to cross while the red hand was flashing, but of course a law abiding driver should yield in this situation. I expect many human drivers would choose to turn anyway as long as the pedestrians were far enough away as they appeared to be. But FSD should have stopped.
- BugsJustFindMe 5y agoIt is illegal to turn right there without signaling though. > What I'm speculating happened here is the new obstacle detection failed and decided that the road ahead was blocked Know what would be able to tell if the road ahead is blocked? Literally any kind of three dimensional sensing that isn't just single frame image object classification. It's a real shame that no one knows how to do that...
- modeless 5y agoNot signaling in advance for what the car likely considers to be an emergency maneuver is a pretty nitpicky criticism. The problem is the unnecessary emergency maneuver, not the lack of signaling. As for lidar, well lidar isn't perfect either. You might get the same behavior from a lidar car if a trash bag happened to be blown into the road ahead by the wind. All types of sensors rely on machine learning to classify their observations. Lidar isn't a shortcut that eliminates that machine learning dependency. Finally, calling what Tesla is doing "single frame image object classification" really betrays your bias here. That hasn't been an accurate description of the system for a long time.
- lmilcin 5y ago"Not perfect". Yes, killing pedestrians is not perfect. I mean, seriously. You are too fucking optimistic guys. Go back to the drawing board. If your program can do that you are way off from the target. You can't write software on which innocent lives depend like you would write a startup AI showcase.
- rndmind 5y agoJust to clarify here. no pedestrians were hit, and no one died. I am curious how many people died on the first airplanes and commercial airline travel ( from 1900 - 1950's ) Do you suggest that the government should have made commercial airline travel illegal until there were zero accidents?
- AnimalMuppet 5y agoNo, lmilcin didn't suggest that. And your analogy is imperfect. How many people died on the ground from airplane crashes from 1900-1950? Probably, not very many. (Why does the difference matter? The passengers got on the plane, knowing - at least approximately - what the risks were. The people on the ground didn't voluntarily accept any risk.)
- lmilcin 5y agoI assume everybody understands there WILL be deaths due to AI making some kind of mistake. It is just a consequence of billions of cars making trillions of trips. But at the very least we should have a clear, reliable solution for all the easy cases. Let's summarize: - clear, sunny day -- check, - but not blinded by the sun -- check, - slow speeds -- check, - clearly marked intersection -- check, - pedestrians not acting stupid -- check, - not trying to avoid hitting another vehicle -- check. Just make sure under no circumstances the car is allowed to turn into a path of a pedestrians like that. And if it can't reliably detect pedestrians or their path in perfect conditions like these I can only conclude it is far from being ready.
- void_mint 5y ago> No, lmilcin didn't suggest that. They posted this: > Yes, killing pedestrians is not perfect. In a thread about a video in which an autonomous vehicle did not kill any pedestrians.
- tiahura 5y agoMaybe I’m missing something but what I saw is common with human drivers (myself included) as well. I.e. pedestrian at corner sees car and stops, driver sees pedestrian and proceeds into turn because pedestrian is stopped at corner, pedestrian doesn’t wait and starts crossing while driver is still turning because they’re in a hurry, driver panics and slams brakes. Just seems like life at the corner of an intersection in a busy city to me?
- 88j88 5y agoJust wondering how much of an increase do insurance companies charge for cars with autopilot?
- jazzyjackson 5y agoI was thinking of this too when watching one of the videos testing whether their car would weave through concrete pillars. if you record yourself colliding with a concrete wall because you were testing beta software, does insurance pay out?
- tzs 5y agoI wonder how well it does with stopping for school buses? Here in Washington I see people getting that wrong all the time, usually in the direction of stopping when they do not have to. The rule is that when a school bus has its flashers on and/or its stop sign out you stop unless: 1. You are going in the opposite direction, and 2. The road is either a divided road (separated by a median or a barrier) or has at least 3 lanes. I see people going the opposite direction often stop on three or four lane roads. (The reason you do not have to stop on a 3+ lane road or divided road is that the law requires that if school has students from both sides of such a road the school must have separate bus stops on each side, so that no child has to cross such a road. School bus stops can only serve both sides of a road if the road is 2 or fewer lanes). (I've not been able to find a definitive answer on whether or not a road with one lane in each direction and a center turn lane counts as a 3 lane road).
- bagels 5y agoIt's low likelihood that it handles that. There are so many videos of it not handling even more mundane things, like this one, or the ones showing it trying to run in to a road closed sign.
- kaminar 5y agoIntentional hands off the wheel before vehicle accident should be worse than DUI conviction. Death caused by same should be 2nd degree murder.
- dude1234 5y agoThe comments here are funny. Having lived in Seattle, owning 2 teslas and actually watching more than this single clip of a beta software: this is deliberate testing in a crazy corner case of a road. The driver is nervous because of the pillers. I use navigate on autopilot (FSD for the masses) literally every day and it navigates multiple highways, every day. It's awesome. Please continue skepticism and never partake, more room for me at the superchargers.
- Woodi 5y agoAttempts to prohibit repairing things is stupid and it is stupid on regulators part to not seeing it progresses. But allowing live-testing auto-drive technology on public roads is beyound that. Police should just pull over anyone who turn this on and is on the move and revoke driver driving license. Simple as that. "Oh, but it works for me..." is not how public security standards are created.
- Arnt 5y agoI think you're holding Tesla to a perhaps unreachable standard here. Before I go on, I'd like to digress and come out as a Waymo fanboi. I've been that ever since I read about a demonstration involving the software developers walking in front of the car, blindfolded. I admire that attitude, I really do. They put their feet where it matters. But back to Tesla. What other than public roads is similar to public roads, in the senses necessary for testing driving capability? I know Waymo tests its software on recordings of public roads, but those recordings were made on public roads too. Maybe Tesla does the same, I don't know one way or the other. It's not an answer, because this approach doesn't avoid public roads, it only extracts more value from each test drive. It would be possible to set up a test area. Maybe rent one of those "towns" the film studios have, and use stunt pedestrians, also from the film studios. I worry that those stunt pedestrians wouldn't be really like pedestrians, though, so this too doesn't replace street tests, with the real variety of real people. These things can supplant public tests. It would be possible to regulate this and say, "software may be used for tests on public tests only when the same version of the same software has been run on one million km of recorded test drives" or such. But as far as I can see, not replace.
- loeg 5y ago> I know Waymo tests its software on recordings of public roads, but those recordings were made on public roads too. Recording a human driver on public roads is very different from testing a novel software product on public roads.
- Arnt 5y agoI've been told that Waymo (and before that, Google) recorded those traces using employees in the driver's seat, but mostly with the computer making the decisions, not the human. But suppose it was collected by a human. In that case, the first time the software is run completely (all sensors, all software, everything run together) is in the final production deployment. Is that better than testing the very same software earlier, in a beta setting which is identical to production except that sensor data is collected for the development process?
- simion314 5y agoIMO each update should be considered a completely different new software, you would need to get again approval to put the shit on the roads. This is not 2020s video game industry where you ship buggy software as done and do monthly patches that fix some shit and break other shit.
- cblconfederate 5y agoReads more and more like a video game