4 ms·

Here's the winning car from the last GC: http://www.tartanracing.org http://www.tartanracing.org Lots of LIDARs, lots of cameras.

toomuchtodo 10y ago
"Stanley", the VW Touareg that won the 2005 grand challenge, also used LIDAR (but fused with camera imagery). In the event that the fused sensor data reduced confidence, the vehicle would slow and lean more heavily on the LIDAR sensors. "Additional guidance data was provided by a video camera used to observe driving conditions out to eighty meters (beyond the range of the LIDAR) and to ensure room enough for acceleration." "Stanley was characterized by a machine learning based approach to obstacle detection. Data from the LIDARs was fused with images from the vision system to perform more distant look-ahead. If a path of drivable terrain could not be detected for at least 40 meters in front of the vehicle, speed was decreased and the LIDARs were used to locate a safe passage." This computer vision processing was performed with 1.6ghz Pentium M processors 11 years ago. Is it so far fetched Nvidia parallel processing hardware can't do better with CCDs than LIDAR?
- argonaut 10y ago
  This is a research problem, not a compute problem. Directly trying to replicate what LIDAR gives you (SLAM, a 3d map of your current environment) using just cameras is still unsolved today. And it is an active area of research.
  - snovv_crash 10y ago
    No, it is solved and there is commercial software that will do this for you. Look at Pix4D, AgiSoft, DroneDeploy, etc. The problem is that it typically doesn't run in real-time, and scales badly with number of images.
    - robotresearcher 10y ago
      So it's not a solved problem then. Apart from realtime there is a robustness problem that the offline reconstruction systems don't suffer from too badly. Missing data and poor reconstructions are inconvenient for an offline system but could be deadly in a car.
      - snovv_crash 10y ago
        Parent said it is a research problem, not a compute problem. I asserted it is a compute problem because we already have robust 3D reconstruction from cameras, but it doesn't run in real-time. If you can come up with a 500GHz CPU with a GB of cache, the problem is solved. It is a compute problem, that we may be able to bypass with research.
        argonaut 10y ago
        1) "Scales badly with the number of images" seems like a research problem to me. 2) In theory almost everything is "just" a compute problem (just enumerate or search over all possibilities - of course this may not finish for millenia). But if Moore's law says the compute is 20 years away, I think it's reasonable to call it a research problem.
  - toomuchtodo 10y ago
    I agree it's a research project. Who better to perform said project than Tesla? They have 100k+ vehicles on the road being trained by drivers at a rate of over 2 million miles per day.
    - deleted 10y ago
      [deleted]
    - argonaut 10y ago
      Tesla does not have the world's experts in visual SLAM. Those people are in academia, or at Google/Microsoft. The 100k car/driver thing is a red herring. Steering angle / brake / acceleration info is not the kind of data you need for this task. Big useless data is still useless data, even if it's big. The right kind of data, if this is to be treated as an ML problem, is a 3D model of the environment (this would be the label), along with camera data (this would be the data). Ironically you need LIDAR to get those labels. It bears noting that the state of the art in visual SLAM is not ML based (it's really difficult to get the "ML" problem I just described to actually work - and it's not for lack of data, Google probably has petabytes of this data from their cars).
      - greglindahl 10y ago
        I thought that the data Tesla is collecting is things like "places where drivers always steer unexpectedly" or "places where drivers always stop but our system didn't detect a stop sign". Those are labels.
        argonaut 10y ago
        Those are not the labels for the problem I was talking about (visual SLAM). Those are labels for a different problem.
agildehaus 10y ago
The last Grand Challenge was in 2007. A lot has changed since then. Tesla's prototypes are doing what that car did, and quite a bit more, without any LIDAR. Not to say LIDAR isn't essential. I think LIDAR will get cheap enough that it'll just be silly not to have it.
- deleted 10y ago
  [deleted]
- argonaut 10y ago
  Tesla is doing highway driving and some turns without LIDAR. The point of LIDAR is to help you parse a complicated urban scene (and so far what Tesla has are demo videos of urban driving). Of course you don't need LIDAR for the highway.

toomuchtodo 10y ago

"Stanley", the VW Touareg that won the 2005 grand challenge, also used LIDAR (but fused with camera imagery). In the event that the fused sensor data reduced confidence, the vehicle would slow and lean more heavily on the LIDAR sensors. "Additional guidance data was provided by a video camera used to observe driving conditions out to eighty meters (beyond the range of the LIDAR) and to ensure room enough for acceleration." "Stanley was characterized by a machine learning based approach to obstacle detection. Data from the LIDARs was fused with images from the vision system to perform more distant look-ahead. If a path of drivable terrain could not be detected for at least 40 meters in front of the vehicle, speed was decreased and the LIDARs were used to locate a safe passage." This computer vision processing was performed with 1.6ghz Pentium M processors 11 years ago. Is it so far fetched Nvidia parallel processing hardware can't do better with CCDs than LIDAR?

argonaut 10y ago
This is a research problem, not a compute problem. Directly trying to replicate what LIDAR gives you (SLAM, a 3d map of your current environment) using just cameras is still unsolved today. And it is an active area of research.
- snovv_crash 10y ago
  No, it is solved and there is commercial software that will do this for you. Look at Pix4D, AgiSoft, DroneDeploy, etc. The problem is that it typically doesn't run in real-time, and scales badly with number of images.
  - robotresearcher 10y ago
    So it's not a solved problem then. Apart from realtime there is a robustness problem that the offline reconstruction systems don't suffer from too badly. Missing data and poor reconstructions are inconvenient for an offline system but could be deadly in a car.
    - snovv_crash 10y ago
      Parent said it is a research problem, not a compute problem. I asserted it is a compute problem because we already have robust 3D reconstruction from cameras, but it doesn't run in real-time. If you can come up with a 500GHz CPU with a GB of cache, the problem is solved. It is a compute problem, that we may be able to bypass with research.
      - argonaut 10y ago
        1) "Scales badly with the number of images" seems like a research problem to me. 2) In theory almost everything is "just" a compute problem (just enumerate or search over all possibilities - of course this may not finish for millenia). But if Moore's law says the compute is 20 years away, I think it's reasonable to call it a research problem.
- toomuchtodo 10y ago
  I agree it's a research project. Who better to perform said project than Tesla? They have 100k+ vehicles on the road being trained by drivers at a rate of over 2 million miles per day.
  - deleted 10y ago
    [deleted]
  - argonaut 10y ago
    Tesla does not have the world's experts in visual SLAM. Those people are in academia, or at Google/Microsoft. The 100k car/driver thing is a red herring. Steering angle / brake / acceleration info is not the kind of data you need for this task. Big useless data is still useless data, even if it's big. The right kind of data, if this is to be treated as an ML problem, is a 3D model of the environment (this would be the label), along with camera data (this would be the data). Ironically you need LIDAR to get those labels. It bears noting that the state of the art in visual SLAM is not ML based (it's really difficult to get the "ML" problem I just described to actually work - and it's not for lack of data, Google probably has petabytes of this data from their cars).
    - greglindahl 10y ago
      I thought that the data Tesla is collecting is things like "places where drivers always steer unexpectedly" or "places where drivers always stop but our system didn't detect a stop sign". Those are labels.
      - argonaut 10y ago
        Those are not the labels for the problem I was talking about (visual SLAM). Those are labels for a different problem.

agildehaus 10y ago

The last Grand Challenge was in 2007. A lot has changed since then. Tesla's prototypes are doing what that car did, and quite a bit more, without any LIDAR. Not to say LIDAR isn't essential. I think LIDAR will get cheap enough that it'll just be silly not to have it.

deleted 10y ago
[deleted]
argonaut 10y ago
Tesla is doing highway driving and some turns without LIDAR. The point of LIDAR is to help you parse a complicated urban scene (and so far what Tesla has are demo videos of urban driving). Of course you don't need LIDAR for the highway.