4 ms·

As the editor of the post, I'd love to get technical feedback and happy to answer any questions.

by jhonovich 13y ago

sp332 13y ago
Why is there a bright outline around the letters? It looks like the photos have been sharpened.
- jhonovich 13y ago
  I am not sure what you mean by 'bright outline'. We added red rectangles around the lowest discernible line. Are you referring to that? We have not sharpened or enhanced the images. They are straight from the cameras, we simply digitally zoomed and cropped the portion of the eye chart to make it easier for people to see the results.
  - sp332 13y ago
    For example, in the "1080p - 160 lux" photo, at the bottom of the E there is a brighter line. Edit: according to my color picker, the color of the chart is generally 8e9fb9 but goes up to becae4 at the bottom of the E.
    - PavlovsCat 13y ago
      If was shot in JPEG, the camera might have sharpened it a bit before even saving the image.
quux 13y ago
Were the cameras saving raw images? Looks like JPEG artifacts are visible in some if the images.
matt_kantor 13y ago
More information about the cameras would be nice. How do their lens sizes compare to the human eye? What kind of post-processing happens in the camera (if any)? What are the characteristics of the CCDs used? Etc.
chuckledog 13y ago
Did all of the cameras tested have the same size lens? A physically large lens should have better low light performance, regardless of the # of megapixels.
gagege 13y ago
The whole experiment seemed strange. Some of the lines you said were unreadable, I could read. Also, some of the ones you said were readable, were unreadable to me. Seems pretty subjective.
nrp 13y ago
I hate to be negative on interesting experiments, but this test is almost useless. The noise of the sensor and the JPEG compression of the camera almost overwhelm the actual resolution in the test images. Additionally, the eye gets a larger amount of information over time during fixation, while the camera is only exposed once, with the resulting aliasing of pixels against the structure of the pattern.
jerf 13y ago
1. As others sugges, if you have a RAW setting on your cameras, you should use that. If you don't, you should probably get one that does. 2. I'm going to disagree with the naysayers who say this is useless. You can partially defang them by pointing out that while this doesn't fully test a camera against a human eye, it is a reasonable test on its own merits and generally answers the question that people are actually asking when they ask "what's the resolution of the human eye?" (After all, the difference in capabilities goes both ways; sure, human eyes are a great deal more sensitive on many measures, but would someone please show me their eye's 10x optical zoom? (Hello to the 2040s!)) A further observation that "megapixels" is not useful would be called for, though. We're really comparing angular resolution for a certain set of settings. But we can accept "megapixels" as a slang term if we also fix certain other camera parameters to ones common to consumer cameras in the market. 3. Using eye charts is a good idea, I think. I've seen comparisons that try to compare rod density on the human eye, but that's really hard. I'd consider adding a note that standardized eye charts measure what humans can really see rather than try to compare two extremely different hardware platforms directly.
- jhonovich 13y ago
  We are using IP cameras, that's our area of research / focus. As such, those types of cameras so not have a raw option. We used H.264 with default quantization. We could have certainly used MJPEG, as these types of cameras support that but in our preliminary testing, it made no difference. That said, I do agree that a completely uncompressed, raw, image could make a modest difference. For our domain, IP cameras, we just don't deal with raw images.
analog31 13y ago
Among the many hats that I wear, one is optics designer. A useful resolution test is to take pictures of alternating black and white bars, then use some sort of image analysis software to measure the contrast ratio, then draw a graph of contrast versus "frequency" which is inversely proportional to the spacing of the bars. You could define the resolution as the frequency or spacing where contrast drops to some arbitrary value such as 50 percent. Contrast won't depend on light level unless the camera is doing some interesting post-processing. It might be worthwhile to Google "USAF test chart" and look at what pops up under Images, because folks have conjured up test patterns that are useful for determining the contrast of an imaging system in one exposure. What I think might be tricky is to come up with an objective definition of resolution or contrast that is amenable to both physical and physiological measurement. [edited to add "inversely"]

As the editor of the post, I'd love to get technical feedback and happy to answer any questions.

sp332 13y ago

Why is there a bright outline around the letters? It looks like the photos have been sharpened.

jhonovich 13y ago
I am not sure what you mean by 'bright outline'. We added red rectangles around the lowest discernible line. Are you referring to that? We have not sharpened or enhanced the images. They are straight from the cameras, we simply digitally zoomed and cropped the portion of the eye chart to make it easier for people to see the results.
- sp332 13y ago
  For example, in the "1080p - 160 lux" photo, at the bottom of the E there is a brighter line. Edit: according to my color picker, the color of the chart is generally 8e9fb9 but goes up to becae4 at the bottom of the E.
  - PavlovsCat 13y ago
    If was shot in JPEG, the camera might have sharpened it a bit before even saving the image.

quux 13y ago

Were the cameras saving raw images? Looks like JPEG artifacts are visible in some if the images.

matt_kantor 13y ago

More information about the cameras would be nice. How do their lens sizes compare to the human eye? What kind of post-processing happens in the camera (if any)? What are the characteristics of the CCDs used? Etc.

chuckledog 13y ago

Did all of the cameras tested have the same size lens? A physically large lens should have better low light performance, regardless of the # of megapixels.

gagege 13y ago

The whole experiment seemed strange. Some of the lines you said were unreadable, I could read. Also, some of the ones you said were readable, were unreadable to me. Seems pretty subjective.

nrp 13y ago

I hate to be negative on interesting experiments, but this test is almost useless. The noise of the sensor and the JPEG compression of the camera almost overwhelm the actual resolution in the test images. Additionally, the eye gets a larger amount of information over time during fixation, while the camera is only exposed once, with the resulting aliasing of pixels against the structure of the pattern.

jerf 13y ago

1. As others sugges, if you have a RAW setting on your cameras, you should use that. If you don't, you should probably get one that does. 2. I'm going to disagree with the naysayers who say this is useless. You can partially defang them by pointing out that while this doesn't fully test a camera against a human eye, it is a reasonable test on its own merits and generally answers the question that people are actually asking when they ask "what's the resolution of the human eye?" (After all, the difference in capabilities goes both ways; sure, human eyes are a great deal more sensitive on many measures, but would someone please show me their eye's 10x optical zoom? (Hello to the 2040s!)) A further observation that "megapixels" is not useful would be called for, though. We're really comparing angular resolution for a certain set of settings. But we can accept "megapixels" as a slang term if we also fix certain other camera parameters to ones common to consumer cameras in the market. 3. Using eye charts is a good idea, I think. I've seen comparisons that try to compare rod density on the human eye, but that's really hard. I'd consider adding a note that standardized eye charts measure what humans can really see rather than try to compare two extremely different hardware platforms directly.

jhonovich 13y ago
We are using IP cameras, that's our area of research / focus. As such, those types of cameras so not have a raw option. We used H.264 with default quantization. We could have certainly used MJPEG, as these types of cameras support that but in our preliminary testing, it made no difference. That said, I do agree that a completely uncompressed, raw, image could make a modest difference. For our domain, IP cameras, we just don't deal with raw images.

analog31 13y ago

Among the many hats that I wear, one is optics designer. A useful resolution test is to take pictures of alternating black and white bars, then use some sort of image analysis software to measure the contrast ratio, then draw a graph of contrast versus "frequency" which is inversely proportional to the spacing of the bars. You could define the resolution as the frequency or spacing where contrast drops to some arbitrary value such as 50 percent. Contrast won't depend on light level unless the camera is doing some interesting post-processing. It might be worthwhile to Google "USAF test chart" and look at what pops up under Images, because folks have conjured up test patterns that are useful for determining the contrast of an imaging system in one exposure. What I think might be tricky is to come up with an objective definition of resolution or contrast that is amenable to both physical and physiological measurement. [edited to add "inversely"]