3 ms·

The trick for level 5 is learning the mapping between the lidar point cloud and the video stream. It’s the best of both worlds.

pgodzin 7y ago
Tesla doesn't have have a lidar point cloud at all
- adamnemecek 7y ago
  That’s the point. You train with lidar, deploy with cameras.
djebdbebeejrrn 7y ago
The trick for level 5 is creating artificial general intelligence. In other words, level 5 will almost certainly not happen any time in the next 50 years. Musk is a con artist and Karpathy is one of his enablers.
ojn 7y ago
That falls apart as soon as the map and the real world deviates and you need to drive based on what’s in front of you. Lidar helps you spot obstructions, but won’t tell you what they are and won’t help you figure out what to do to avoid them. Want an example? Cruise’s first real world demo got stuck behind a simple taco truck in downtown SF.
- antpls 7y ago
  The parent meant : gather training data with Lidar and Camera, then build a model with that data to learn to reconstruct a 3D space only from Camera data, and then embed that model in the cars. Tesla is already using a model to rebuild a 3D space from Camera data only, the parent suggests to improve the quality of the transformation with high quality 3D representations from Lidar. It's deep learning all the way down.
ben_w 7y ago
That might be a trick. It’s not the trick human brains use. It might be equivalent to the way that when we say we want to “look at” a thing, we often also want to touch it. https://kitsunesoftware.wordpress.com/2017/07/27/why-do-people-look-by-touching/ https://kitsunesoftware.wordpress.com/2017/07/27/why-do-peop...
sheeshkebab 7y ago
One could also train a car driving model driving in grand theft auto... but are all these tricks really what level 5 is about? I doubt
Geee 7y ago
2D to 3D transform is simple trigonometry (using stereo / motion) and should be possible to learn without lidar. I think this is already a solved problem. One option though is to add lidars in random Teslas (e.g. 1/1000) to help with the labeling / learning.
- adamnemecek 7y ago
  The field is called photogrammetry. It's not...precise. Point clouds have a much richer structure.

pgodzin 7y ago

Tesla doesn't have have a lidar point cloud at all

adamnemecek 7y ago
That’s the point. You train with lidar, deploy with cameras.

djebdbebeejrrn 7y ago

The trick for level 5 is creating artificial general intelligence. In other words, level 5 will almost certainly not happen any time in the next 50 years. Musk is a con artist and Karpathy is one of his enablers.

ojn 7y ago

That falls apart as soon as the map and the real world deviates and you need to drive based on what’s in front of you. Lidar helps you spot obstructions, but won’t tell you what they are and won’t help you figure out what to do to avoid them. Want an example? Cruise’s first real world demo got stuck behind a simple taco truck in downtown SF.

antpls 7y ago
The parent meant : gather training data with Lidar and Camera, then build a model with that data to learn to reconstruct a 3D space only from Camera data, and then embed that model in the cars. Tesla is already using a model to rebuild a 3D space from Camera data only, the parent suggests to improve the quality of the transformation with high quality 3D representations from Lidar. It's deep learning all the way down.

ben_w 7y ago

That might be a trick. It’s not the trick human brains use. It might be equivalent to the way that when we say we want to “look at” a thing, we often also want to touch it. https://kitsunesoftware.wordpress.com/2017/07/27/why-do-people-look-by-touching/ https://kitsunesoftware.wordpress.com/2017/07/27/why-do-peop...

sheeshkebab 7y ago

One could also train a car driving model driving in grand theft auto... but are all these tricks really what level 5 is about? I doubt

Geee 7y ago

2D to 3D transform is simple trigonometry (using stereo / motion) and should be possible to learn without lidar. I think this is already a solved problem. One option though is to add lidars in random Teslas (e.g. 1/1000) to help with the labeling / learning.

adamnemecek 7y ago
The field is called photogrammetry. It's not...precise. Point clouds have a much richer structure.