3 ms·

I agree that, ultimately, vision and some IMUs will be all that's needed for an as-perfect-as-possible self-driving cars. Your statement on phone cameras being

I agree that, ultimately, vision and some IMUs will be all that's needed for an as-perfect-as-possible self-driving cars. Your statement on phone cameras being better than the human eye isn't true today, though. The human eye has FAR better dynamic range than any camera built to date, at any price point. This matters a great deal on sunny, cloudless days in a city with tall buildings casting dark shadows, for example. Just like how the biological system of "stupid sensor, smart thinking" helps biological organisms, that's what is going to need to be done for computer vision as well. About 1/3 of the human brain is dedicated to vision processing in some way. Think about that a second. One THIRD of the best, most powerful organic computer known is required for us to see what we see and we are still fooled by the things human eyes can be tricked by. It's going to take a lot of neural network training to duplicate that. Fortunately, the vision skills required to drive are a subset of overall vision capabilities.

ajross 8y ago
> The human eye has FAR better dynamic range than any camera built to date, at any price point. Not at all true. All you need to do is take your camera into a dark room for proof. It can take useful pictures in environments you can't see. And with some manual control and safety precautions (seriously, don't actually do this) you'll note you can shoot useful photos of things that are very near bright sources like the sun where your eyes would be completely useless (and irrevocably damaged, again do not try this). What you're complaining about isn't dynamic range, it's exposure control. A human brain, again, makes much better decisions about what parts of the environment are "important" when setting the aperture (iris). So the stuff you want to see is visible, where the camera is mostly just going to guess that the center of the frame is what you want and will routinely leave stuff over or underexposed. Your last paragraph has the correct analysis but the wrong conclusion. It's the vision processing that makes the difference. The optical systems of a semiconductor camera absolutely are better, so a control system based on optics can absolutely be better.
- naikrovek 8y ago
  Dynamic range is the ability to see things in both direct sunlight AND in dark shadows at the same time. You need to be able to see pedestrians that are both directly lit by the sun and in the darkest shadows in the same frame of video. No long exposures, no multiple exposures with varying exposure -- a single frame. I don't know of any 24-32-bit per channel sensors, and that's what you'd need. We agree on vision processing. I thought I made it clear that it's a strong "backend" to vision (the brain or compute behind it) that makes it work so well, and that will be the case with good self-driving cars as well. I must have misstated something.
  - ajross 8y ago
    You're fooling yourself if you think your eyes can do that. They can't. More to the point, your brain is fooling you, by synthesizing a perception of a reality, the same way it does to make you think the stuff in your peripheral vision is as sharp as the things in the middle. Vertebrate eyes don't have 24 bits of sensitivity, that's just insane. Rod cells are neurons, they either fire or they don't, can fire at most at about 10 Hz, and you have about a million of them at most across your whole field of vision. Do the math. What you say isn't even physically possible.
    - naikrovek 8y ago
      You're correct that the brain fills in a lot that isn't there, making you think your vision is a lot better than it is. It doesn't change much, though, mostly it fills in gaps. 10Hz? Simply not true. Fighter pilots can identify aircraft shown on a screen for 1/250th of one second. Regular schmoes can see the difference between 30Hz and 60Hz video easily. I am not arguing about pixel count of a camera vs the center of vision of human eyes. I'm saying that... Nevermind. I've already said it several times and apparently you have a PhD in all things vision.
      - Robotbeat 8y ago
        Being able to recognize aircraft on a screen for 1/250th of a second just means the human eye/brain has persistence, not actual useful 250Hz bandwidth.
- pedrocr 8y ago
  You're swapping dynamic range and exposure control. Your examples of dark rooms and bright light sources are examples of great exposure control, not dynamic range. Dynamic range is the difference in light intensity from pure black to pure white in your sensor. Modern sensors are also extremely good at dynamic range though, much better than film and apparently much better than our eyes as well. From a quick search our eyes are at 10 stops of range (2^10 ratio from white to black) and modern sensors have gone past 15 stops and are pushing on 20 apparently, which is just amazing.
tigershark 8y ago
It’s absolutely false. The eye simulates the dynamic range by continuously changing the pupil size. If you take a camera with a lens that can go from f/1.4 to f/32 you have a much much bigger dynamic range counting that you can also modify the exposure time. Your comparison between the eye with his changing pupil size and a single image taken at a fixed aperture value and a fixed exposure is totally incorrect.

ajross 8y ago

> The human eye has FAR better dynamic range than any camera built to date, at any price point. Not at all true. All you need to do is take your camera into a dark room for proof. It can take useful pictures in environments you can't see. And with some manual control and safety precautions (seriously, don't actually do this) you'll note you can shoot useful photos of things that are very near bright sources like the sun where your eyes would be completely useless (and irrevocably damaged, again do not try this). What you're complaining about isn't dynamic range, it's exposure control. A human brain, again, makes much better decisions about what parts of the environment are "important" when setting the aperture (iris). So the stuff you want to see is visible, where the camera is mostly just going to guess that the center of the frame is what you want and will routinely leave stuff over or underexposed. Your last paragraph has the correct analysis but the wrong conclusion. It's the vision processing that makes the difference. The optical systems of a semiconductor camera absolutely are better, so a control system based on optics can absolutely be better.

naikrovek 8y ago
Dynamic range is the ability to see things in both direct sunlight AND in dark shadows at the same time. You need to be able to see pedestrians that are both directly lit by the sun and in the darkest shadows in the same frame of video. No long exposures, no multiple exposures with varying exposure -- a single frame. I don't know of any 24-32-bit per channel sensors, and that's what you'd need. We agree on vision processing. I thought I made it clear that it's a strong "backend" to vision (the brain or compute behind it) that makes it work so well, and that will be the case with good self-driving cars as well. I must have misstated something.
- ajross 8y ago
  You're fooling yourself if you think your eyes can do that. They can't. More to the point, your brain is fooling you, by synthesizing a perception of a reality, the same way it does to make you think the stuff in your peripheral vision is as sharp as the things in the middle. Vertebrate eyes don't have 24 bits of sensitivity, that's just insane. Rod cells are neurons, they either fire or they don't, can fire at most at about 10 Hz, and you have about a million of them at most across your whole field of vision. Do the math. What you say isn't even physically possible.
  - naikrovek 8y ago
    You're correct that the brain fills in a lot that isn't there, making you think your vision is a lot better than it is. It doesn't change much, though, mostly it fills in gaps. 10Hz? Simply not true. Fighter pilots can identify aircraft shown on a screen for 1/250th of one second. Regular schmoes can see the difference between 30Hz and 60Hz video easily. I am not arguing about pixel count of a camera vs the center of vision of human eyes. I'm saying that... Nevermind. I've already said it several times and apparently you have a PhD in all things vision.
    - Robotbeat 8y ago
      Being able to recognize aircraft on a screen for 1/250th of a second just means the human eye/brain has persistence, not actual useful 250Hz bandwidth.
pedrocr 8y ago
You're swapping dynamic range and exposure control. Your examples of dark rooms and bright light sources are examples of great exposure control, not dynamic range. Dynamic range is the difference in light intensity from pure black to pure white in your sensor. Modern sensors are also extremely good at dynamic range though, much better than film and apparently much better than our eyes as well. From a quick search our eyes are at 10 stops of range (2^10 ratio from white to black) and modern sensors have gone past 15 stops and are pushing on 20 apparently, which is just amazing.

tigershark 8y ago

It’s absolutely false. The eye simulates the dynamic range by continuously changing the pupil size. If you take a camera with a lens that can go from f/1.4 to f/32 you have a much much bigger dynamic range counting that you can also modify the exposure time. Your comparison between the eye with his changing pupil size and a single image taken at a fixed aperture value and a fixed exposure is totally incorrect.