3 ms·

How does vision work at night?! RGB Cameras dont see anything...

mensetmanusman 2y ago
Apparently they have been surprised at how few photons are required to see for these sensors. They are skipping the image computer vision step and going from photons to car control in as few layers as possible.
- smaddox 2y ago
  Apparently they're using this CMOS sensor: https://www.onsemi.com/products/sensors/image-sensors/ar0136at https://www.onsemi.com/products/sensors/image-sensors/ar0136... It's not an event camera, so it's very much taking images, which are then being processed by computer vision algorithms. Event cameras seem more viable than CMOS sensors for autonomous vehicle applications in the absence of LIDAR. CMOS dynamic range and response isn't as good as the human eye. LIDAR+CMOS is considerably better in many ways.
  - astrange 2y ago
    Humans can also decide they want to get a better look at something and move their head (or block out the sun with their hands) which cameras generally don't do.
  - iknowstuff 2y ago
    Nah thats old. They now use https://www.sony-semicon.com/files/62/pdf/p-15_IMX490.pdf https://www.sony-semicon.com/files/62/pdf/p-15_IMX490.pdf and they run multiexposure on every frame. Stupid high dynamic range
    - smaddox 2y ago
      Oh, interesting. 120db is much better than what I thought possible with CMOS sensors. Thats competetive with the human eye. Well, that definitely changes my opinion on how feasible/competitive camera-only autonomous driving can be.
      - iknowstuff 2y ago
        Next time you’re facing blinding direct sunlight, pull out your iPhone and take a picture/video. It’s a piece of cake for it. And it has to do far more post processing to make a compelling jpeg/heic for humans. Tesla can just dump the data from the sensor from short&long exposures straight into the neural net.
porphyra 2y ago
The Tesla cameras have surprisingly good dynamic range and can see stuff incredibly well even when human eyes struggle.
plun9 2y ago
How do people see at night while driving?

Apparently they have been surprised at how few photons are required to see for these sensors. They are skipping the image computer vision step and going from photons to car control in as few layers as possible.

smaddox 2y ago
Apparently they're using this CMOS sensor: https://www.onsemi.com/products/sensors/image-sensors/ar0136at https://www.onsemi.com/products/sensors/image-sensors/ar0136... It's not an event camera, so it's very much taking images, which are then being processed by computer vision algorithms. Event cameras seem more viable than CMOS sensors for autonomous vehicle applications in the absence of LIDAR. CMOS dynamic range and response isn't as good as the human eye. LIDAR+CMOS is considerably better in many ways.
- astrange 2y ago
  Humans can also decide they want to get a better look at something and move their head (or block out the sun with their hands) which cameras generally don't do.
- iknowstuff 2y ago
  Nah thats old. They now use https://www.sony-semicon.com/files/62/pdf/p-15_IMX490.pdf https://www.sony-semicon.com/files/62/pdf/p-15_IMX490.pdf and they run multiexposure on every frame. Stupid high dynamic range
  - smaddox 2y ago
    Oh, interesting. 120db is much better than what I thought possible with CMOS sensors. Thats competetive with the human eye. Well, that definitely changes my opinion on how feasible/competitive camera-only autonomous driving can be.
    - iknowstuff 2y ago
      Next time you’re facing blinding direct sunlight, pull out your iPhone and take a picture/video. It’s a piece of cake for it. And it has to do far more post processing to make a compelling jpeg/heic for humans. Tesla can just dump the data from the sensor from short&long exposures straight into the neural net.

porphyra 2y ago

The Tesla cameras have surprisingly good dynamic range and can see stuff incredibly well even when human eyes struggle.

plun9 2y ago

How do people see at night while driving?