7 ms·
Tesla FSD Beta Barely Avoids Crashing into Pole
- rvz 5y agoFSD = Fools Self Driving.
- GenerocUsername 5y agoI wonder if that pole has ever been struck by a human driver turning right?
- jeffbee 5y agoWhy is this flagged? We just don't criticize Tesla? The whole video on YouTube is a disaster. The car turns left across facing traffic on a red left arrow. Easiest machine-vision problem you can imagine: chroma-keyed directional signal.
- libpcap 5y agoThe downside of not having a LIDAR.
- basicplus2 5y agoDoes not 'barely avoid crashing'.. it was going to crash! except driver took over and steered it away from pole!!
- WrtCdEvrydy 5y agoThis also plays into the theory that all those numbers on safety only count when Autopilot is enabled. If you take control, Autopilot is off so you're the one who crashed instead.
- pcbro141 5y agoThe FSD team must be pissed at Elon putting so much public pressure on them to roll out the beta more widely. They have to know releasing it widely in this state is going to be a nightmare.
- ajross 5y agoDoes this really qualify as a nightmare? It's not even an accident. I mean, bad PR is bad PR; there's literally a whole cottage industry (c.f. the linked twitter account) dedicated to spinning stories against Tesla. But it's reasonably clear that this system, even in this state, isn't causing accidents. I think an open beta like this is absolutely appropriate. Lord knows I'm enjoying watching it (but not using it yet, didn't make the Safety Score cut).
- rich_sasha 5y agoIt was pretty close! For me it also shows how much in the ball you need to be while supposedly relaxing on FSD. This really was a split-second decision, out of nowhere, with no warning. And, presumably, since in FSB you’re supposed to be alert all the time, insurance will point the finger at you for driving into a lamppost. Not Elon’s fault at all.
- Isinlor 5y agoTesla is saying that "It may do the wrong thing at the worst time, so you must (...) pay extra attention to the road. (...) Use FSD only if you will pay constant attention to the road, and be prepared to act immediately, (...)" Anyone supervising it for longer than 15 min will realize that Tesla means it and that it is not some lawyer speak. The FSD Beta will become dangerous when it will start driving flawlessly for 50h, enough to make people complacent, and crash into a pedestrian on 51h.
- dhooper 5y ago.
- BoorishBears 5y agoEvery time I say this people lose their heads: Tesla's FSD/Enhanced AP cannot be safer than a human driver + passive aids like LKA and AEB. The statistics that constantly get paraded about AP are vs all cars, not just cars with passive aids. It's simple enough: If FSD would catch a hazard with the sensor suite, so would the passive aids. But if the sensor suite would miss a hazard, it's not an issue unless the human does too when it comes to passive aids. If the human misses it and the sensor suite misses it, so will FSD. The problem is this: With FSD the car will actively do the wrong thing in response to the missed hazard instead of doing nothing. Now the human has to correct after the wrong action has already been taken. - So if you visualize a venn diagram of accident conditions, the ones prevented by FSD will fit inside the ones prevented by passive aids, since FSD is actively creating accidents where they couldn't exist before.
- tux1968 5y agoThat assumes that the human driver responds appropriately to the passive aids. But an electronic system that never gets tired or distracted may eventually perform much better and more consistently in this regard.
- BoorishBears 5y agoThat's the trick people keep falling for... Tesla FSD still requires a fully attentive driver, so it has almost nothing in common with the systems you're describing. It sounds like a small difference, but it completely upends 99% of the assumptions you can make about its performance. - For example you can't response to passive aids, you can't use FSD. Look at the video we just watched, would a tired or distracted driver have been able to make that correction? - Passive aids even handle tired drivers _better_ than FSD. Toyota's aids will warn the driver they are drowsy if the passive aids are intervening too often. They've done this for years now, even in their most basic cars. FSD is assuming computer control with human backup, so the human backup isn't being engaged enough to make calls that well. At best it can use proxies like wheel jiggling and gaze.
- ajross 5y agoI don't see how that follows. You're assuming the human always makes the right decision, which is wrong by construction in this particular situation as the problem being solved is the fact that drivers mess up. I mean, you're right that if you take a putative autonomy solution and add another redudant sensor system (not even a human) that you'll catch situations the first wouldn't. But if that system is then allowed to demand control, then you open the possibility of mistakes in the "supervisor layer". So you're still limited by the best available quality of one system. Specific example: go watch highway crash videos on youtube for a while. A very common situation is a "late stop" situation on the highway where a car fails to see the one its following stop in time. Most drivers just slam on the brake and hit the car ahead, but a significant fraction swerve instead. And those swerves cause rollovers and jacknifes and all sorts of worse accidents. So... how does your hypothetical "human driver + passive aids" prevent the driver from incorrectly swerving at high speeds?
- aeternum 5y agoFSD clearly has some work to do here, but it's also quite surprising how many human drivers end up driving onto light rail tracks. Seems like better markings would really help.
- jeffbee 5y agoBetter drivers would help more.
- supernova87a 5y ago"Yeah, we don't need Lidar at all!" except when it's going to kill you.
- ajross 5y agoPeople don't tend to discuss this much, but in fact Lidar has somewhat limited horizontal resolution and a ton of noise in the signal. Linear scan-aligned things like poles don't show up well until they're quite close. It's obviously likely it would have seen it before this car did, but Lidar cars would have the same kind of behavior of "Why didn't it see THAT?", just in a different regime. (The best theory I've seen presented for this particular bug, FWIW, is that it was the shadows and training data at fault. Teslas can, obviously, recognize street lights. But this one evaded detection at a distance because it was inside the dark shadow and presumably the contract detection failed to resolve it. And once it was in the near field, the recognizers don't have as much training data and failed to see it.)
- ra7 5y ago> People don't tend to discuss this much, but in fact Lidar has somewhat limited horizontal resolution and a ton of noise in the signal. This was probably true in 2016, but not anymore. Waymo’s 5th gen lidar and Argo AI’s lidar all show massive performance and resolution improvements. They have been game changers. Anyone not using lidar in 2021 is just handicapping themselves, as we are seeing with Tesla FSD performance.
- kjksf 5y agoCan you quantify the improvements for us? As far as I know the specs for Waymo's 5th generation lidar are not publicly available. Can you link to them? How much work did you do with Waymo's 5th generation lidar? Or any lidar for that matter? Do you have any qualifications to claim that a lidar that isn't even available to you is capable of doing a very specific task?
- ra7 5y ago
- TedShiller 5y agoThat's why it's called artificial intelligence as opposed to actual intelligence
- toothrot 5y agoThe scary part isn't that it's about to hit a pole, it's that it jumped the curb onto a sidewalk!
- atatatat 5y agoNot if you're the occupant of the vehicle. </DevilsAdvocacy>
- ra7 5y agoIt’s not a sidewalk. It’s a railroad track for the VTA train in San Jose. FSD has no idea it’s not a drivable area. Guess what other technology Tesla has shunned that would’ve completely eliminated this issue? HD maps.
- redis_mlc 5y agoThere are offramp/VTA/Highway 101 junctions in Sunnyvale (roughly near the former Blue Cube) that humans can't decode, let alone any car. As a pedestrian, I had to watch for a couple of light cycles to figure out where the lanes were. There's multiple versions of roadway markings in seemingly random directions, some leading to cement columns. I think the new Apple Tesla driver got killed near there.
- bdamm 5y agoThey are building maps of drivable areas. It may not be HD, but it certainly would apply in this scenario.
- bagels 5y agoI hope they don't use where the fsd decides to drive as truth data for those maps.
- toothrot 5y agoThe pole literally has pedestrian buttons on it. I'm fairly certain that humans are intended to push them. Maps might help, though!
- justapassenger 5y agoIt's always amazing to me, how so many techies, who personally know how overhyped AI is (it's amazing at moving you from 0 to maybe 50% solution, then with tons of extra work, maybe to 80%, but to 100%? just not possible), are buying FSD-hype claims from Tesla.
- ajross 5y agoIt seems very unlikely that this was a bug in the "AI" layer. The object in question was a streetlight, which Teslas see with effective perfection already, in shipping versions of the autopilot. There is no doubt at all that a Tesla can recognize that object. The best analysis I've seen is that this is an imaging problem. The lamp in the shadow didn't have enough contrast to show up and be presented to the ML recognizers (and, probably, that when it did it was too late to see the object in totality and there wasn't a recognizer for "vertical pole in the very near field").
- rasz 5y agoIf you see a streetlight post and then sun bounces off of another cars window and momentarily blinds you - did that street post stop existing? If you crash into it, did you crash because you got blinded? or because you "forgot" what you kept seeing a second ago? Is closing your eyes altering reality?
- ajross 5y agoI'm not following. Persistence of the model is a different problem. FSD does that too, there are lots of examples. The issue here seems to be that the car never saw the post at all (you can see on the screen that it never gets rendered, though it's for the cross traffic and I don't know whether FSD beta draws lights that it thinks aren't for you).
- andy-x 5y agoCan AI feel suicidal (or is it homicidal) after being tested for so many hours on San Jose streets?
- deleted 5y ago[deleted]
- endianswap 5y agoI wonder if the driver would've been able to successfully swerve if they had the new rectangular steering wheel.
- cromwellian 5y agoNot having RADAR + LIDAR here is the problem IMHO. You need multi-sensors, relying on purely visual is an issue, and when humans run into visual ambiguity (there was a ton of tree shadow on the ground), we instinctively slow down or stop. Mission #0 for self driving is to never collide head on with something. Radar car systems that automatically apply braking from 20 years ago could achieve this. IMHO, forward facing radar running without AI and straight up deterministic algorithms should be able to take control away from the ML systems when a collision in imminent.
- ajross 5y agoNo radar collision avoidance system in production would have seen that pole. They're tuned very strongly to avoid seeing stationary objects (to avoid signaling for candy wrappers and manhole covers). Lidar might have, but it's much thinner than you think and lidar has much (like an order of magnitude) poorer horizontal resolution than cameras. Really, this is fixable. The car can see streetlights. I mean, duh. This wasn't an out-of-regime situation. There's a bug, mostly likely in the contrast control layer (the lamp was in a shadow). Bugs can be fixed, that's what beta is for. This is going to be a popular genre ("Did you see what that Tesla did?") for a while as the new capabilities appear in public. And then they'll be fixed, and we'll go back to arguing about other stuff.
- Ancapistani 5y agoThe obstacle avoidance sensors on my quadcopter wouldn’t have let me hit that pole. They are just optical cameras, and work at speeds of ~22mph and below. It seems nuts to me that Tesla can’t do better than a ~$1,500 quadcopter.
- ajross 5y ago> It seems nuts to me that Tesla can’t do better than a ~$1,500 quadcopter. That's probably because you've mischaracterized the issue. It's a streetlight. Obviously Teslas see streetlights. Go for a ride in a friends car (not even one with FSD Beta) and look at the screen. You'll see streetlights rendered by the hundreds. It's basically perfect. It failed here, not because it couldn't see "a pole" but because of a much more specific bug. Specific bugs can, almost always, be fixed.
- bmitc 5y agoIt didn't barely avoid crashing into the pole. The driver did. The car was actively steering into the pole. Here's the full video: https://youtu.be/bbyNg9kYEq4 https://youtu.be/bbyNg9kYEq4 The guys seem to be Tesla fans, despite needing to intervene multiple times to avoid crashes. Also, I noticed that it displays concrete pillars as cones. Also, is it really just a LIDAR issue? LIDAR would help with the pole, but it wouldn't help with the car wanting to turn onto railroad tracks, changing its mind, and then swerving back for the railroad tracks. It remains unbelievable to me that this is allowed on the roads. I actively avoid Tesla's on the road.
- avereveard 5y agoIf perfect weather and no traffic is considered a stress test, I wonder what it'll do comes the wonder holiday shopping craze
- travisgriggs 5y agoFSD = Full Self Driving I kinda figured something like that from the context, but not hanging on the days and lives of our Tesla future, this is the first time I heard this particular acronym. Can we (HN community) please recommit to spell things out on first use unless the abbreviation is long and established and unambiguous?