5 ms·
The inputs to FSD are: 7 cameras x 36fps x 5Mpx x 30s 48kHz audio Nav maps and route for next few miles 100Hz kinematics (speed, IMU, odometry,
by smallmancontrov 9mo ago
The inputs to FSD are:
7 cameras x 36fps x 5Mpx x 30s
48kHz audio
Nav maps and route for next few miles
100Hz kinematics (speed, IMU, odometry, etc)
Source: https://youtu.be/LFh9GAzHg1c?t=571 https://youtu.be/LFh9GAzHg1c?t=571
- ambicapter 9mo agoSo if they’re already “fusioning” all these things, why would LIDAR be any different?
- smallmancontrov 9mo agoTesla went nothing-but-nets (making fusion easy) and Chinese LIDAR became cheap around 2023, but monocular depth estimation was spectacularly good by 2021. By the time unit cost and integration effort came down, LIDAR had very little to offer a vision stack that no longer struggled to perceive the 3D world around it. Also, integration effort went down but it never disappeared. Meanwhile, opportunity cost skyrocketed when vision started working. Which layers would you carve resources away from to make room? How far back would you be willing to send the training + validation schedule to accommodate the change? If you saw your vision-only stack take off and blow past human performance on the march of 9s, would you land the plane just because red paint became available and you wanted to paint it red? I wouldn't completely discount ego either, but IMO there's more ego in the "LIDAR is necessary" case than the "LIDAR isn't necessary" at this point. FWIW, I used to be an outspoken LIDAR-head before 2021 when monocular depth estimation became a solved problem. It was funny watching everyone around me convert in the opposite direction at around the same time, probably driven by politics. I get it, I hate Elon's politics too, I just try very hard to keep his shitty behavior from influencing my opinions on machine learning.
- kanbara 9mo agodepth estimation is but one part of the problem— atmospheric and other conditions which blind optical visible spectrum sensors, lack of ambient (sunlight) and more. lidar simply outperforms (performs at all?) in these conditions. and provides hardware back distance maps, not software calculated estimation
- gibolt 9mo agoLidar fails worse than cameras in nearly all those conditions. There are plenty of videos of Tesla's vision-only approach seeing obstacles far before a human possibly could in all those conditions on real customer cars. Many are on the old hardware with far worse cameras
- Mawr 9mo agoInteresting, got any links? Sounds completely unbelievable, eyes are far superior to the shitty cameras Tesla has on their cars.
- deleted 9mo ago[deleted]
- jellojello 9mo agoYea.. not holding my breath on links to superman tesla cameras performing better than eyes
- dham 9mo agoThere's a misconception that what people see and what the camera sees is similar. Not true at all. One day when it's raining or foggy, have some record the driving, through the windshield. You'll be very surprised. Even what the camera displays on the screen isn't what it's actually "seeing".
- kranke155 9mo agoAlways thought the case was for sensor redundancy and data variety - the stuff that throws off monocular depth estimation might not throw off a lidar or radar.
- 7e 9mo agoMonocular depth estimation can be fooled by adversarial images, or just scenes outside of its distribution. It's a validation nightmare and a joke for high reliability.
- gibolt 9mo agoIt isn't monocular though. A Tesla has 2 front-facing cameras, narrow and wide-angle. Beyond that, it is only neural nets at this point, so depth estimation isn't directly used; it is likely part of the neural net, but only the useful distilled elements.
- smallmancontrov 9mo agoI never said it was. I was using it as a lower bound for what was possible.
- magicalist 9mo ago> but monocular depth estimation was spectacularly good by 2021 It's still rather weak and true monocular depth estimation really wasn't spectacularly anything in 2021. It's fundamentally ill posed and any priors you use to get around that will come to bite you in the long tail of things some driver will encounter on the road. The way it got good is by using camera overlap in space and over time while in motion to figure out metric depth over the entire image. Which is, humorously enough, sensor fusion.
- smallmancontrov 9mo agoIt was spectacularly good before 2021, 2021 is just when I noticed that it had become spectacularly good. 7.5 billion miles later, this appears to have been the correct call.
- quanto 9mo agoWhat are the techniques (and the papers thereof) that you consider to be spectacularly good before 2021 for depth estimation, monocular or not? I do some tangent work from this field for applications in robotics, and I would consider (metric) depth estimation (and 3D reconstruction) starting to be solved only by 2025 thanks to a few select labs. Car vision has some domain specificity (high similarity images from adjacent timestamps, relatively simpler priors, etc) that helps, indeed.
- rswail 9mo agoIt doesn't solve the "Coyote paints tunnel on rock" problem though.
- verelo 9mo agoBetter than I expected. So this was 3 days ago, is this for all previously models or is there a cut off date here?
- ChicagoDave 9mo agoFog, heavy rain, heavy snow, people running between cars or from an obstructed view… None of these technologies can ever be 100%, so we’re basically accepting a level of needless death. Musk has even shrugged off FSD related deaths as, “progress”.
- smallmancontrov 9mo agoHumans: 70 deaths in 7 billion miles FSD: 2 deaths in 7 billion miles Looks like FSD saves lives by a margin so fat it can probably survive most statistical games.
- hn_acc1 9mo agoIs that the official Tesla stat? I've heard of way more Tesla fatalities than that..
- ChicagoDave 9mo agoThis is absolutely a Musk defender. FSD and Tesla related deaths are much higher. https://www.tesladeaths.com/index-amp.html https://www.tesladeaths.com/index-amp.html
- smallmancontrov 9mo agoAutopilot is the shitty lane assist. FSD is the SOTA neural net. Your link agrees with me: > 2 fatalities involving the use of FSD
- ChicagoDave 9mo agoTesla sales are dead across the world. Cybertruck is a failure. Chinese EVs are demonstrably better. No one wants these crappy cars anymore.
- Fricken 9mo agoI don't know what he's on about. Here's a better list: https://en.wikipedia.org/wiki/List_of_Tesla_Autopilot_crashes#:~:text=There%20have%20been%20multiple%20fatal,instance%2C%20the%20motorcyclist%20was%20killed. https://en.wikipedia.org/wiki/List_of_Tesla_Autopilot_crashe...