3 ms·
The “fleet learning” pipeline Elon described at Autonomy Day doesn’t actually exist in practice. They have some hardcoded heuristic triggers that capture snippe
by jowday 5y ago
The “fleet learning” pipeline Elon described at Autonomy Day doesn’t actually exist in practice. They have some hardcoded heuristic triggers that capture snippets of data to be annotated by human labelers to train perception models off of. The bottleneck is labeling/engineering.
Autonomous vehicles by and large aren’t a data problem anyways - it’s a robotics problem conditioned on the output of ML models for perception and prediction. The majority of the work is just implementing all of the bizarre edge cases inside of the robotics stack.
The idea of tackling this with an e2e ML model is a pipe dream touted by people that aren’t familiar with the space or trying to hype up their approach. Whenever this approach is attempted teams will very quickly realize how untenable it is and return to implementing individual robotics modules.
tl;dr there’s nothing special about Tesla’s approach to AVs or machine learning and you shouldn’t expect them to leapfrog the competition because of some nebulous ML-based advantage.
Coming from someone who has seen the interiors of both the FSD beta and the stack at an actual AV company.
- rcMgD2BwE72F 5y ago>The idea of tackling this with an e2e ML model is a pipe dream touted by people that aren’t familiar with the space or trying to hype up their approach. Whenever this approach is attempted teams will very quickly realize how untenable it is and return to implementing individual robotics modules. Except Andrej Karpathy. Why dumb down if they know it isn't working, now that they have all the financial resources to pivot and overtake the other manufacturers (who have not achieve anything anyway)? It makes no sense, unless Karpathy does not know what he's doing. Edit: the argument that others don't agree is not convincing because Tesla should not have succeeded by others' standards either.
- jowday 5y agoThey’re not tackling it with an end to end model - all of the ML at Tesla is used for perception (and some prediction - I think there’s a cut-in prediction head running), which then feeds into traditional robotics algorithms, like I described in the parent comment. If you listen to any of AK’s talks he only really talks about perception and he’s basically said that right now none of their planning is based on ML models. This is backed up by what reverse engineers have found in their Tesla’s.
- rcMgD2BwE72F 5y agoYes, but we've all heard about Dojo https://youtu.be/Ucp0TTmvqOE?t=8809 https://youtu.be/Ucp0TTmvqOE?t=8809 https://www.inverse.com/innovation/tesla-dojo-self-driving-vehicles https://www.inverse.com/innovation/tesla-dojo-self-driving-v..., and Software 2.0 https://karpathy.medium.com/software-2-0-a64152b37c35 https://karpathy.medium.com/software-2-0-a64152b37c35
- jowday 5y agoAfaict dojo is just a training asic integrated into a cluster, just like other big tech companies have had for years. If anything at all it’ll just mean a lower cloud or compute bill for Tesla. It’s telling that Elon is the one that likes to mention Dojo and Karpathy is generally quiet about it - it’s not actually a quantum leap in ML capabilities.
- briefcomment 5y ago> "data to be annotated by human labelers" Could you speed up labelling by using a combination of eye tracker cameras, road cameras, and brain signal sensors on real drivers? That way you could quickly understand what road footage corresponds with certain brain signals. Assuming brain signals are easier to label.