3 ms·

The current SOTA in monocular “spatial sensing” (3D object detection) is just much less reliable compared to LiDAR-only systems. You can’t conclude that spatia

The current SOTA in monocular “spatial sensing” (3D object detection) is just much less reliable compared to LiDAR-only systems. You can’t conclude that spatial sensing isn’t a problem because it didn’t crash with a monocular camera. The test is whether it accurately detects all relevant actors/objects in the scene and does so reliably. From a couple FSD videos I’ve seen, the issues center around perception (failing to get the right lane lines, certain detections disappearing briefly, etc.)

simondotau 4y ago
> You can’t conclude that spatial sensing isn’t a problem because it didn’t crash with a monocular camera. The test is whether it accurately detects all relevant actors/objects in the scene and does so reliably. Indeed, which is why Tesla have put a lot of effort into showing a visual representation of its object detection. Unlike many (most) other players in this space, Tesla isn't afraid to let people see what their system is capable of in literally any situation their customers wish to drive to. Most of their competitors are hiding behind L4 which, because of how machine learning works, makes systems appear better than they are. Overfitting is a great way to make rapid gains, but much of it is illusory. > From a couple FSD videos I’ve seen, the issues center around perception (failing to get the right lane lines, certain detections disappearing briefly, etc.) Even if true, LiDAR wouldn't help as it cannot see painted lane lines and is extremely weak at object identification. Looking at the most recent FSD beta (10.12+) as shown on YouTube by real customers, it's quite clear that detection is already damn close to exceptional. When FSD does fail, it's usually either out of an abundance of caution (it is extremely cautious around pedestrians) or because the path planner wasn't making a good decision. It's almost never because the car has failed to see something that LiDAR might have.
- upbeat_general 4y ago
  > Overfitting is a great way to make rapid gains, but much of it is illusory. I'm not sure why you think this is the case for L4. Do you mean since it can be geo/weather restricted the learning task is easier? Surely weather plays a large role but I don't see why other L4 companies necessarily overfit more (also how does overfitting provide rapid gains?). If you overfit when training for your L4 system...it's not an L4 system. > Even if true, LiDAR wouldn't help as it cannot see painted lane lines and is extremely weak at object identification. True, but: A) Again, the test is reliable detection which is simply out of reach atm from single-sensor input B) HD maps help a lot with this issue since the lane lines become less important
  - simondotau 4y ago
    When you limit your service area, it becomes feasible to train your models on pretty much (or literally) every road in the service area. Every single permutation of weird intersection it will ever come across. If your L4 car only ever has to drive in San Francisco, it doesn't matter that this ML model would have no hope when introduced to Salt Lake City. The model never has to spend a moment excluding any of the millions of potential things it has never been trained on and will never see. Both the ease of data collection and the absence of contextual noise makes L4 machine learning an order of magnitude easier. Can you point me towards any recent FSD Beta video where there was an important failure of object detection?

simondotau 4y ago

> You can’t conclude that spatial sensing isn’t a problem because it didn’t crash with a monocular camera. The test is whether it accurately detects all relevant actors/objects in the scene and does so reliably. Indeed, which is why Tesla have put a lot of effort into showing a visual representation of its object detection. Unlike many (most) other players in this space, Tesla isn't afraid to let people see what their system is capable of in literally any situation their customers wish to drive to. Most of their competitors are hiding behind L4 which, because of how machine learning works, makes systems appear better than they are. Overfitting is a great way to make rapid gains, but much of it is illusory. > From a couple FSD videos I’ve seen, the issues center around perception (failing to get the right lane lines, certain detections disappearing briefly, etc.) Even if true, LiDAR wouldn't help as it cannot see painted lane lines and is extremely weak at object identification. Looking at the most recent FSD beta (10.12+) as shown on YouTube by real customers, it's quite clear that detection is already damn close to exceptional. When FSD does fail, it's usually either out of an abundance of caution (it is extremely cautious around pedestrians) or because the path planner wasn't making a good decision. It's almost never because the car has failed to see something that LiDAR might have.

upbeat_general 4y ago
> Overfitting is a great way to make rapid gains, but much of it is illusory. I'm not sure why you think this is the case for L4. Do you mean since it can be geo/weather restricted the learning task is easier? Surely weather plays a large role but I don't see why other L4 companies necessarily overfit more (also how does overfitting provide rapid gains?). If you overfit when training for your L4 system...it's not an L4 system. > Even if true, LiDAR wouldn't help as it cannot see painted lane lines and is extremely weak at object identification. True, but: A) Again, the test is reliable detection which is simply out of reach atm from single-sensor input B) HD maps help a lot with this issue since the lane lines become less important
- simondotau 4y ago
  When you limit your service area, it becomes feasible to train your models on pretty much (or literally) every road in the service area. Every single permutation of weird intersection it will ever come across. If your L4 car only ever has to drive in San Francisco, it doesn't matter that this ML model would have no hope when introduced to Salt Lake City. The model never has to spend a moment excluding any of the millions of potential things it has never been trained on and will never see. Both the ease of data collection and the absence of contextual noise makes L4 machine learning an order of magnitude easier. Can you point me towards any recent FSD Beta video where there was an important failure of object detection?