3 ms·

More of certain types of light, importantly. They don’t have the dynamic range of colors to capture all the colors our eyes can see, but they can pick up other

by dangus 2y ago

More of certain types of light, importantly. They don’t have the dynamic range of colors to capture all the colors our eyes can see, but they can pick up other types of light that we can’t. This fact was pointed out in a lot of reviews of the Vision Pro. It’s just about good enough to make you forget you’re in augmented reality but the world looks more dull and lifeless because your eyes see more vibrant colors than small digital sensors.

speedylight 2y ago
Yep, phone cameras can pick up IR light which is very useful for testing whether a TV remote works or not!
- amelius 2y ago
  Any reason why evolution didn't give us IR vision too?
  - burnte 2y ago
    Not knowing if the remote batteries were dead wasn't a big driver of reproduction a million years ago, mostly.
  - michaelt 2y ago
    "Infra-red" covers a bunch of wavelengths. 'Near Infra-red' is very close to visible light, and is what's used by remote controls and picked up by cell phone camera sensors. It's around the 0.75–1.4 μm range. On the other hand, thermal cameras are sensitive to 'Long-wavelength infrared' which is around 8–15 μm wavelength. Marketers of things like CCTV cameras love to sow confusion about these things, as NIR-sensitive cameras are extremely cheap while thermal cameras are comparatively expensive. Humans are not sensitive to NIR because for the vast majority of human existence, any time there was NIR light there was also an abundance of normal visible light, due to a little thing called "the sun"
    - what-the-grump 2y ago
      Idk, I think it would be nice to have non shit night vision.
      - LeifCarrotson 2y ago
        Non-shit night vision, the kind where warm things glow at night, requires LWIR sensors that are at a temperature lower than the warm thing (otherwise the sensor itself glows). NIR sensitivity does not improve night vision. That mostly requires a "tapetum lucidum" or reflective layer, more rods than cones (less helpful when you want to see colors in daylight) or just larger, more biologically expensive and vulnerable eyes than necessary.
        dTal 2y ago
        > NIR sensitivity does not improve night vision Your comment and its grandparent are both excellent, but I would like to nitpick a bit here. Technically, NIR sensitivity does improve night vision through the simple mechanism of "detect more wavelengths = detect more total light". It's not thermal imaging, but it is the reason that CCTV cameras often mechanically move their IR filter out of the way when the light falls below some threshold - at that point, sensitivity is more important than color fidelity, so they accept all the light they can get, even nonvisible. Of course this is also often coupled with NIR illumination LEDs.
  - radford-neal 2y ago
    Since we're warm-blooded animals, maybe it's because IR vision would be swamped by the emissions of the eye itself?
    - amelius 2y ago
      Doesn't the eye lens solve that by concentrating multiple rays onto one point?
- babypuncher 2y ago
  Newer smartphones have an IR filter in the lens assembly so this doesn't really work anymore
  - runjake 2y ago
    It works with my iPhone 15 Pro. I do not see any visible light emitting from the remote. This is in a pitch dark room. Edit: proof, taken in a semi-lit room for clarity: https://imgur.com/a/63FbXjQ https://imgur.com/a/63FbXjQ Edit 2: Tested on iPhone 15 Pro selfie camera. It detects IR at the same intensity. Edit 3: Same as above with a Pixel 8. Edit 4: I now have my office involved, this is fun. Same with a Samsung S24. Final Edit: OK, I just went through about 20 colleague's phones, which are various mixes of iPhone and Android, new and old. Testing the front and back camera, every camera on every phone saw the IR light. If a smartphone camera does not see IR light, this appears to be the exception, not the rule. OK, back to work! These are all US devices, in case that matters. We don't seem to have an agency that regulates light emissions from cell phones.
    - michaelt 2y ago
      Some phones have an IR filter only on the main camera, not on the selfie camera. So if you really want this to work, try the selfie camera instead.
      - runjake 2y ago
        See my updated comment above. I just tested a bunch of phones, and all of the cameras front and back could see the IR light. I acknowledge that I may be the only person in the world that is this interested in this fact.
        alexbock 2y ago
        I've always been fascinated by this topic as well. As a further experiment, you may be interested to know that these IR lights can pass straight through red wine that looks totally dark and opaque to the human eye. I took some photos to demonstrate this with a DSLR with the IR filter removed here [1], but you can test this yourself by using a smartphone to look at the IR light of a TV remote with a glass of red wine in between them. [1] https://alexbock.github.io/blog/nir-water-red-wine-comparison/index.html https://alexbock.github.io/blog/nir-water-red-wine-compariso...
doctoboggan 2y ago
I don't think this is true. The less than real feeling from the AVP is more likely from the display than the camera. The human eye is capable of a wider color gamut and (probably more importantly) a much wider dynamic range of intensity than the AVP display is capable of producing.
- Tagbert 2y ago
  Yes. The cameras on the VP are optimized for weight and size and don't have as good color sensitivity as larger lenses. The microOLED screens on the VP also do not have the range than a normal OLED would. It's a limitation of the current tech.
crazygringo 2y ago
> They don’t have the dynamic range of colors to capture all the colors our eyes can see Are you sure about that? My understanding is that digital cameras can capture an extremely wide color gamut -- the vibrant colors -- but that the extra information is necessarily thrown away when encoded in sRGB or P3 in your image/video file. Because we don't have many displays which are capable of showing those ultra-saturated colors, so we don't waste bits on storing them in files either. P3 is an obvious improvement over sRGB that Apple products mostly use now. If you process RAW files directly from the sensor, I'm pretty sure you get a much wider gamut. What I'm not entirely sure about is how that gamut maps to the colors our eyes can see -- it's not going to be exactly the same coverage. So I'm not sure where it goes beyond human eye sensitivity, or where it doesn't quite reach it, and to what extent this depends on the sensor technology (e.g. what's in your phone vs. an expensive DSLR vs. a professional cinema camera). Ultimately it's going to come down to the exact precise shape of the frequency curve in each of the R, G and B filters in the color filter array, and how they can be mathematically translated to reflect the human eye's [1, 2]. (And so color vibrancy limitations with the Vision Pro are absolutely going to be coming from the displays before they come from the cameras.) [1] https://en.wikipedia.org/wiki/Color_filter_array#Image_sensor_overview https://en.wikipedia.org/wiki/Color_filter_array#Image_senso... [2] https://en.wikipedia.org/wiki/Color_management#Color_transformation https://en.wikipedia.org/wiki/Color_management#Color_transfo...
mrandish 2y ago
Yes! I came specifically to post "yeah but... dynamic range". As someone heavily into video production and imaging in general, we're still a long way from capturing the overall fidelity of the human visual system. The current state of what high-end consumer gear delivers to end users in HDR and WCG (wide color gamut) is still quite disappointing compared to the marketing claims and technical specs. It requires a surprising amount of expertise, effort and money to assemble, calibrate and feed content to a high-end home theater sufficient to correctly deliver even the (currently modest) full capabilities of 4k HDR10+ / DolbyVi$ion. And the home theater content ecosystem is still a mess of mixed, partial implementation of supposed standards. Implementation of HDR/WCG for web media is currently even less evolved. Despite many newer, high-end laptop, computer and phone screens having decent capabilities, inconsistent file formats, browsers, drivers and OS implementations seem far away from this "just working" for end users.

by dangus 2y ago

speedylight 2y ago

Yep, phone cameras can pick up IR light which is very useful for testing whether a TV remote works or not!

amelius 2y ago
Any reason why evolution didn't give us IR vision too?
- burnte 2y ago
  Not knowing if the remote batteries were dead wasn't a big driver of reproduction a million years ago, mostly.
- michaelt 2y ago
  "Infra-red" covers a bunch of wavelengths. 'Near Infra-red' is very close to visible light, and is what's used by remote controls and picked up by cell phone camera sensors. It's around the 0.75–1.4 μm range. On the other hand, thermal cameras are sensitive to 'Long-wavelength infrared' which is around 8–15 μm wavelength. Marketers of things like CCTV cameras love to sow confusion about these things, as NIR-sensitive cameras are extremely cheap while thermal cameras are comparatively expensive. Humans are not sensitive to NIR because for the vast majority of human existence, any time there was NIR light there was also an abundance of normal visible light, due to a little thing called "the sun"
  - what-the-grump 2y ago
    Idk, I think it would be nice to have non shit night vision.
    - LeifCarrotson 2y ago
      Non-shit night vision, the kind where warm things glow at night, requires LWIR sensors that are at a temperature lower than the warm thing (otherwise the sensor itself glows). NIR sensitivity does not improve night vision. That mostly requires a "tapetum lucidum" or reflective layer, more rods than cones (less helpful when you want to see colors in daylight) or just larger, more biologically expensive and vulnerable eyes than necessary.
      - dTal 2y ago
        > NIR sensitivity does not improve night vision Your comment and its grandparent are both excellent, but I would like to nitpick a bit here. Technically, NIR sensitivity does improve night vision through the simple mechanism of "detect more wavelengths = detect more total light". It's not thermal imaging, but it is the reason that CCTV cameras often mechanically move their IR filter out of the way when the light falls below some threshold - at that point, sensitivity is more important than color fidelity, so they accept all the light they can get, even nonvisible. Of course this is also often coupled with NIR illumination LEDs.
- radford-neal 2y ago
  Since we're warm-blooded animals, maybe it's because IR vision would be swamped by the emissions of the eye itself?
  - amelius 2y ago
    Doesn't the eye lens solve that by concentrating multiple rays onto one point?
babypuncher 2y ago
Newer smartphones have an IR filter in the lens assembly so this doesn't really work anymore
- runjake 2y ago
  It works with my iPhone 15 Pro. I do not see any visible light emitting from the remote. This is in a pitch dark room. Edit: proof, taken in a semi-lit room for clarity: https://imgur.com/a/63FbXjQ https://imgur.com/a/63FbXjQ Edit 2: Tested on iPhone 15 Pro selfie camera. It detects IR at the same intensity. Edit 3: Same as above with a Pixel 8. Edit 4: I now have my office involved, this is fun. Same with a Samsung S24. Final Edit: OK, I just went through about 20 colleague's phones, which are various mixes of iPhone and Android, new and old. Testing the front and back camera, every camera on every phone saw the IR light. If a smartphone camera does not see IR light, this appears to be the exception, not the rule. OK, back to work! These are all US devices, in case that matters. We don't seem to have an agency that regulates light emissions from cell phones.
  - michaelt 2y ago
    Some phones have an IR filter only on the main camera, not on the selfie camera. So if you really want this to work, try the selfie camera instead.
    - runjake 2y ago
      See my updated comment above. I just tested a bunch of phones, and all of the cameras front and back could see the IR light. I acknowledge that I may be the only person in the world that is this interested in this fact.
      - alexbock 2y ago
        I've always been fascinated by this topic as well. As a further experiment, you may be interested to know that these IR lights can pass straight through red wine that looks totally dark and opaque to the human eye. I took some photos to demonstrate this with a DSLR with the IR filter removed here [1], but you can test this yourself by using a smartphone to look at the IR light of a TV remote with a glass of red wine in between them. [1] https://alexbock.github.io/blog/nir-water-red-wine-comparison/index.html https://alexbock.github.io/blog/nir-water-red-wine-compariso...

doctoboggan 2y ago

I don't think this is true. The less than real feeling from the AVP is more likely from the display than the camera. The human eye is capable of a wider color gamut and (probably more importantly) a much wider dynamic range of intensity than the AVP display is capable of producing.

Tagbert 2y ago
Yes. The cameras on the VP are optimized for weight and size and don't have as good color sensitivity as larger lenses. The microOLED screens on the VP also do not have the range than a normal OLED would. It's a limitation of the current tech.

crazygringo 2y ago

> They don’t have the dynamic range of colors to capture all the colors our eyes can see Are you sure about that? My understanding is that digital cameras can capture an extremely wide color gamut -- the vibrant colors -- but that the extra information is necessarily thrown away when encoded in sRGB or P3 in your image/video file. Because we don't have many displays which are capable of showing those ultra-saturated colors, so we don't waste bits on storing them in files either. P3 is an obvious improvement over sRGB that Apple products mostly use now. If you process RAW files directly from the sensor, I'm pretty sure you get a much wider gamut. What I'm not entirely sure about is how that gamut maps to the colors our eyes can see -- it's not going to be exactly the same coverage. So I'm not sure where it goes beyond human eye sensitivity, or where it doesn't quite reach it, and to what extent this depends on the sensor technology (e.g. what's in your phone vs. an expensive DSLR vs. a professional cinema camera). Ultimately it's going to come down to the exact precise shape of the frequency curve in each of the R, G and B filters in the color filter array, and how they can be mathematically translated to reflect the human eye's [1, 2]. (And so color vibrancy limitations with the Vision Pro are absolutely going to be coming from the displays before they come from the cameras.) [1] https://en.wikipedia.org/wiki/Color_filter_array#Image_sensor_overview https://en.wikipedia.org/wiki/Color_filter_array#Image_senso... [2] https://en.wikipedia.org/wiki/Color_management#Color_transformation https://en.wikipedia.org/wiki/Color_management#Color_transfo...

mrandish 2y ago

Yes! I came specifically to post "yeah but... dynamic range". As someone heavily into video production and imaging in general, we're still a long way from capturing the overall fidelity of the human visual system. The current state of what high-end consumer gear delivers to end users in HDR and WCG (wide color gamut) is still quite disappointing compared to the marketing claims and technical specs. It requires a surprising amount of expertise, effort and money to assemble, calibrate and feed content to a high-end home theater sufficient to correctly deliver even the (currently modest) full capabilities of 4k HDR10+ / DolbyVi$ion. And the home theater content ecosystem is still a mess of mixed, partial implementation of supposed standards. Implementation of HDR/WCG for web media is currently even less evolved. Despite many newer, high-end laptop, computer and phone screens having decent capabilities, inconsistent file formats, browsers, drivers and OS implementations seem far away from this "just working" for end users.