3 ms·

I'm not sure I understand what you mean. I've done my share of video and photo programming on Linux (in particular I've written non-trivial v4l2 drivers for use

by simias 7y ago

I'm not sure I understand what you mean. I've done my share of video and photo programming on Linux (in particular I've written non-trivial v4l2 drivers for use in things like car rear view cameras, where obviously low latency is absolutely critical and you have to resort to tricks like processing the image by slice before you've read the full sensor) but I'm not sure I see how that's relevant for a "point and shoot" type camera. For one thing I don't think any reasonable camera would have the CPU "bitbang" the reading of the sensor's pixel, it will be read and buffered in hardware and set to the system RAM through DMA. We're talking about hundreds of megabytes of raw data per picture here, it's not something you want to memcpy(), especially on an embedded processor. If receiving a tweet is enough to make you miss some pixels then you haven't prioritized your RAM controller correctly or your system is severely overloaded already. The shutter release is independent of course, but it's not something where a few microseconds are going to change anything, especially for a mechanical shutter where you need to move physical parts. If your CPU stalls for 1ms processing a tweet then, again, you have bigger problems. You don't need an RTOS to control a camera shutter. So that leaves us with the problem of latency but even there I fail to see the problem. My girlfriend is a professional photographer and when she shoots she can either get the preview on the camera's screen (but that involves switching to the preview which already usually takes a pair of seconds) or getting the image on a PC connected through USB, which also takes a significant amount of time. Adding or removing 100ms of latency won't change a lot in practice as far as I can tell.

ChrisMarshallNY 7y ago
The "receiving a tweet" thing was really just a strawman. It's more likely to be issues of OS maintenance that could cause the problems. I've had some really relevant experience, here, myself. I won't go into detail (N of the D-A). > it will be read and buffered in hardware and set to the system RAM through DMA. Moving memory, even on the fastest processor, is always a cost. This is especially true, when you look at the pixel count of most cameras (not just DSLRs), these days. Also, you don't just get the memory at once in a big multi-megabyte dump. The sensor needs to be polled. Each cell is queried for its charge state in a linear fashion. There may be ways to access sections in parallel, but I haven't encountered it (It would make sense -I have tended to be on the software side of things, leaving the hardware to the electrical engineering wizards). This is usually dealt with by an ASIC connected directly to the sensor. If you want to have a "software-only" camera (as opposed to one with a dedicated SoC), then you are looking at some really heavy-duty challenges. The idea of a "software-only" camera, is to completely do away with the dedicated SoC, or replace it with one that can be re-flashed. Not so simple. Computational Photography is something that is necessary for consumer-grade cameras, and is increasingly becoming relevant to pros, as well. It takes some real horsepower. The newer iPhones have awesome cameras. I suspect that some of the higher-end Androids are just as good, or better. If so, I will bet that you'll find a custom ASIC in the woodpile. Apple has custom ASICs for image handling, as well as CP. They have an ML chip built into their phones. I am confident that these challenges will be overcome (there's money to be made, and that's always a great driver), but the challenges are real. There are some very, very smart folks that have beat their brains out on this.

by simias 7y ago

ChrisMarshallNY 7y ago

The "receiving a tweet" thing was really just a strawman. It's more likely to be issues of OS maintenance that could cause the problems. I've had some really relevant experience, here, myself. I won't go into detail (N of the D-A). > it will be read and buffered in hardware and set to the system RAM through DMA. Moving memory, even on the fastest processor, is always a cost. This is especially true, when you look at the pixel count of most cameras (not just DSLRs), these days. Also, you don't just get the memory at once in a big multi-megabyte dump. The sensor needs to be polled. Each cell is queried for its charge state in a linear fashion. There may be ways to access sections in parallel, but I haven't encountered it (It would make sense -I have tended to be on the software side of things, leaving the hardware to the electrical engineering wizards). This is usually dealt with by an ASIC connected directly to the sensor. If you want to have a "software-only" camera (as opposed to one with a dedicated SoC), then you are looking at some really heavy-duty challenges. The idea of a "software-only" camera, is to completely do away with the dedicated SoC, or replace it with one that can be re-flashed. Not so simple. Computational Photography is something that is necessary for consumer-grade cameras, and is increasingly becoming relevant to pros, as well. It takes some real horsepower. The newer iPhones have awesome cameras. I suspect that some of the higher-end Androids are just as good, or better. If so, I will bet that you'll find a custom ASIC in the woodpile. Apple has custom ASICs for image handling, as well as CP. They have an ML chip built into their phones. I am confident that these challenges will be overcome (there's money to be made, and that's always a great driver), but the challenges are real. There are some very, very smart folks that have beat their brains out on this.