4 ms·

I've worked on CMOS camera sensors for this market. I'm not sure what's in the newer Tesla right now, but I'm quite sure it's not consumer sensors in there. I b

by audunw 5y ago

I've worked on CMOS camera sensors for this market. I'm not sure what's in the newer Tesla right now, but I'm quite sure it's not consumer sensors in there. I believe Tesla was a customer of one of the products I worked on. First, you have to manufacture to automotive grade standards. So that right there is going to set you back a generation or two. You can't just pick some consumer grade sensor off the shelf. Second, you now have to confirm to ASIL safety standards. A lot of the work we did on these sensors was to build in continuous self testing circuitry. For every frame the sensor would actually inject known values into the physical pixels, read them out and process them like a real picture, and compare with the reference before sending data out to the ISP. I'd also like to know what webcams that aren't insanely expensive have better dynamic range. Is that really true? You list this as the third spec, but this was the most important spec to design for after safety. In part because dynamic range is also important for safety. And it has to be high dynamic range without too many artifacts. The sensor I worked on could do three exposures simultaneously on the same sensor and could either stitch them together on-die or stream out all three exposures to an external ISP. I've never heard of a consumer camera sensor with that kind of capability, but then I don't follow the market much anymore. Sensitivity is obviously also important. I think the bayer pattern was RGBW and sacrificed color accuracy for sensitivity. Higher resolution would compromise sensitivity for a given die size. Resolution and frame rate could be better, but frankly it's about good enough for what they're trying to do.

ithinkso 5y ago
What does ISP mean in this context?
- giobox 5y ago
  In context of camera sensors, commonly means “Image Signal Processor”. https://en.wikipedia.org/wiki/Image_processor https://en.wikipedia.org/wiki/Image_processor Much of the advancement in modern digital cameras is at the ISP layer, for example Apple invest massively here each year to improve the iPhone’s cameras. At the very least, the sensors on the Tesla appear to have no IR filter on key cameras (orange cast on side cameras etc) to improve low light performance. Virtually al consumer grade webcams will have an IR filter, so it’s not quite grab a crappy webcam and shove in the car as some comments suggest! Tesla do also have a custom ISP, it’s part of the custom silicon chip in their standard fit “FSD computer” stack: https://en.wikichip.org/wiki/tesla_(car_company)/fsd_chip https://en.wikichip.org/wiki/tesla_(car_company)/fsd_chip
iamleppert 5y ago
The $200 Logitech cameras have 3-exposure HDR support with high frame rate and sensitivity, including IR. No one knows what the specs of the sensors are in the car but there are physical limits on what you can do with pure electronic exposure control (fixed iris). If the sensors on the Tesla were really that good, why did they blame them on the accident where it could not distinguish between a white tractor trailer and the horizon? In that case, plenty of light was available so it points to saturation or low dynamic range. If the sensor is saturated and has no way to physically limit the amount of light incident on it, there is no amount of clever software or filtering that can provide anything other than noise, but I’m sure you know this. It’s also a function of heat, all that light heats up the sensor and has to be removed and dissipated some how (and quickly) before the next reading is taken. The more heat, the more noise and depending on the noise process model, the less overall range. High sensitivity and high range are conflicting goals and subject to design trade offs that when both are necessary increase the overall system complexity. The whole “humans can drive via vision” thing is a misnomer because human eyes are extraordinarily sensitive, ultra wide angle, stabilized (basically like having a gimbal), accurately temperature controlled via your body, and don’t have a fixed iris. It seems that in order to make that claim, you’d need to first develop a sensor that at least meets or exceeds the specs of the human eye, which no one has really done yet, at least not in an economically and reliable enough package to put on a car. Given enough time, money and equipment it could be done but the sensor would look (and cost) more like something you’d see on an expensive astronomer’s telescope.

by audunw 5y ago

ithinkso 5y ago

What does ISP mean in this context?

giobox 5y ago
In context of camera sensors, commonly means “Image Signal Processor”. https://en.wikipedia.org/wiki/Image_processor https://en.wikipedia.org/wiki/Image_processor Much of the advancement in modern digital cameras is at the ISP layer, for example Apple invest massively here each year to improve the iPhone’s cameras. At the very least, the sensors on the Tesla appear to have no IR filter on key cameras (orange cast on side cameras etc) to improve low light performance. Virtually al consumer grade webcams will have an IR filter, so it’s not quite grab a crappy webcam and shove in the car as some comments suggest! Tesla do also have a custom ISP, it’s part of the custom silicon chip in their standard fit “FSD computer” stack: https://en.wikichip.org/wiki/tesla_(car_company)/fsd_chip https://en.wikichip.org/wiki/tesla_(car_company)/fsd_chip

iamleppert 5y ago

The $200 Logitech cameras have 3-exposure HDR support with high frame rate and sensitivity, including IR. No one knows what the specs of the sensors are in the car but there are physical limits on what you can do with pure electronic exposure control (fixed iris). If the sensors on the Tesla were really that good, why did they blame them on the accident where it could not distinguish between a white tractor trailer and the horizon? In that case, plenty of light was available so it points to saturation or low dynamic range. If the sensor is saturated and has no way to physically limit the amount of light incident on it, there is no amount of clever software or filtering that can provide anything other than noise, but I’m sure you know this. It’s also a function of heat, all that light heats up the sensor and has to be removed and dissipated some how (and quickly) before the next reading is taken. The more heat, the more noise and depending on the noise process model, the less overall range. High sensitivity and high range are conflicting goals and subject to design trade offs that when both are necessary increase the overall system complexity. The whole “humans can drive via vision” thing is a misnomer because human eyes are extraordinarily sensitive, ultra wide angle, stabilized (basically like having a gimbal), accurately temperature controlled via your body, and don’t have a fixed iris. It seems that in order to make that claim, you’d need to first develop a sensor that at least meets or exceeds the specs of the human eye, which no one has really done yet, at least not in an economically and reliable enough package to put on a car. Given enough time, money and equipment it could be done but the sensor would look (and cost) more like something you’d see on an expensive astronomer’s telescope.