11 ms·

I still don't understand why Apple has chosen to include such a sophisticated and expensive sensor of so little use to average consumers in its flagship product

by berta 5y ago

I still don't understand why Apple has chosen to include such a sophisticated and expensive sensor of so little use to average consumers in its flagship products. They're probably setting the stage for some future technology, maybe AR related?

technobabbler 5y ago
IMO phones are so stagnant now that manufacturers just tack on random things to see what sticks. Lidar! Radar! Refresh rate! A 7th camera! Oooh. Who the hell uses any of that stuff? But it lets them keep releasing new versions and charge huge amounts for new flagships. Those gimmicks help to prevent the phones from becoming the next race to the bottom that the PC and cheap Android markets have become.
- bliteben 5y ago
  hopefully thermal imaging next
  - technobabbler 5y ago
    Yeah, that and uv would be cool for looking at plant life. The seek and flir imagers are cool for IR
    - bliteben 5y ago
      UV seems more possible now that I think about it, thermal would present huge issues with export regulations
- sho 5y ago
  > Lidar! Radar! Refresh rate! A 7th camera! [..] Who the hell uses any of that stuff? Well... you're right about the lidar, but I absolutely care about the camera and refresh rate. As a lifelong photography geek I basically upgrade for the camera improvements every year.. Camera, messaging, maps and dating apps are basically the only things I use my phone for these days...
  - technobabbler 5y ago
    That's cool. I'm glad there are still people interested in photography for its own sake instead of posing for Instagram!
  - adonese 5y ago
    What do you suggest for a good camera with a tight budget? I used to have Samsung Galaxy S20 FE and it was just amazing. Trying to buy a new android preferably from flagship killers type of phones
  - saiya-jin 5y ago
    Lidar is actually really useful for portraits, Samsung S22 ultra uses it extensively and can find a single hair strand it doesn't blur compared to the background. Everything looks more realistic compared to pure software processing where I can spot errors quickly. As frequent user my good old Nikon full frame (D750), I am really astonished by output these tiny cameras produce. Sure, its not for big screen/print unless we're talking about sunny day. But for everything else, it makes fantastic portraits. I got fed up with the need to carry big heavy pouch around all the time, and missed way too many pictures of my kids to rely on it anymore. It also sees in the dark much better than I do with my eyes, also thanx to Lidar - the pics I take from night walks where there is basically no light source hundreds meters around and I am in the forest (yes my night walks take me sometimes to interesting places, I normally don't use any light). Very dark scene becomes full of (usually) true colors and details simply invisible to me. 10x zoom on my phone allows me to read signs not readable to my eyesight (digital 'AI' zoom is very useful up to cca 30x). I was skeptical too, and its true full frame is still so far ahead, but at what cost - bulk, weight, the need to spend hours on postprocessing batch of photos instead of quick edits in phone in few seconds.
    - technobabbler 5y ago
      Yeah, portrait mode is basically face ID in reverse and works a lot better than room scanning. How does the lidar help with night shots though? Maybe focusing, but otherwise isn't it just a long exposure with algorithmic corrections?
- amelius 5y ago
  Glad they didn't include a laserpointer.
- smoldesu 5y ago
  > Those gimmicks help to prevent the phones from becoming the next race to the bottom that the PC and cheap Android markets have become. See: it allows them to keep increasing the cost of their devices while simultaneously preventing them from passing the savings from their streamlined manufacturing to the customer.
- andybak 5y ago
  Why is 3d scanning any more of a gimmick than photography? When I travel I scan things for the same reason I photograph things.
  - technobabbler 5y ago
    The lidar plays a trivial role in that. Most of it is just algorithmic photogrammetry. If you turned the laser off you'd probably still get similar results. And it's a gimmick because it's a tiny niche of people who do that, and I wish I didn't have to pay extra for a phone for a lidar. Would take a fingerprint scanner any day. Or a headphone jack.
    - andybak 5y ago
      > Most of it is just algorithmic photogrammetry. If you turned the laser off you'd probably still get similar results. This is simply untrue. Most 3D scanning apps on iOS make no use of photogrammetry or they do it as an alternate mode for smaller objects. I don't know of one 3D scanning app that combines the LIDAR with photogrammetry. They use one or the other depending on the scale of the object you're scanning. > And it's a gimmick because it's a tiny niche of people who do that, It's not the reason Apple added it. They added it for AR and photography. Arguably the former is a gimmick but the latter is future proofing and might end up being a game-changer.
peterburkimsher 5y ago
Could the 3D model be used to interpolate the shape of an animal behind a cage? I'm imagining a panda enclosure, with lots of people taking photos. 2D images are obscured by a line where the cage is. Eyes can correct for it by moving our heads around, but the photos can't. There is an opportunity for cage-removal to be solved with software (guessing, or taking multiple photos and cross-stitching) or simpler hardware (camera on bottom of phone instead of top, use parallax like 2 eyes). But the lidar option might work too.
samvher 5y ago
Why do you think it has so little use? One thing that comes to my mind is portrait background blur - so far computational implementations on smartphones that I’ve seen were not very good, I suspect this can improve effects like that a lot which seems relevant to consumers. On the forward looking side I think app developers can also get a lot of value out of this. Eg (just a thought) changing the way clothes or shoes are bought by letting you scan body parts to determine fit. And this can generate a lot of data feeding into their vision models + what you already mention about AR/VR.
- berta 5y ago
  Well, it doesn't seem to me that Apple has backed this new technology with any significant app that makes use of it. And I don't see any serious app in the apple store that makes use of lidar either.
  - CubsFan1060 5y ago
    I think you dismissed the portrait mode thing, which I think is by far the biggest use. Two other things I love -- "measure". It's very accurate. Also, there are several apps that allow you to scan an area and then send that to people. I've used it to show people an area (my office, a cool brewery I went to, etc). "3d scanner app" is one that I've used. Interestingly, I also used it to scan a piece of furniture I liked, which let me get precise measurements at a later date. Edit: I forgot one super cool use of measure. Let's say you have a basement with an open ceiling. You can use the measure app to, say, measure a pipe. Then, when you walk upstairs, you can still see the pipe in 3D. So you can then understand just exactly under the floor where the pipe is. Very useful for things like pipes and electric wires.
    - berta 5y ago
      The last one is pretty cool indeed. Still, I feel like Apple has still some "one more thing" for lidar in the future
    - Kon-Peki 5y ago
tyingq 5y ago
There's some practical stuff, like mapping a room so that you can see wallpaper or paint choices on your own walls. Or getting clothing sizes more accurately, etc.
deleted 5y ago
[deleted]
nikhizzle 5y ago
Focusing a camera in low light is still not great with traditional methods such as phase detection. LiDAR focus is spot on every time. The quality of the camera is one of the main selling points of the iPhone (Eg. The ad campaign on billboards is “Shot on iPhone”. So I would argue it is a very appropriate feature.
acthrowthrow123 5y ago
I'm iOS dev and few limitation: 1) Biggest one Lidar is only on iPhones Pro in contrast to TrueDepth that after iPhone X landed in all iPhone devices 2) Lidar has very low resolution I wish the added Lidar to all iPhones + combined this with TrueDepth (both in front and back of iphone) - true depth has better resolution but is slower (only 30fps + higher latency) and only works best for distance < 1 meter

by berta 5y ago

technobabbler 5y ago

IMO phones are so stagnant now that manufacturers just tack on random things to see what sticks. Lidar! Radar! Refresh rate! A 7th camera! Oooh. Who the hell uses any of that stuff? But it lets them keep releasing new versions and charge huge amounts for new flagships. Those gimmicks help to prevent the phones from becoming the next race to the bottom that the PC and cheap Android markets have become.

bliteben 5y ago
hopefully thermal imaging next
- technobabbler 5y ago
  Yeah, that and uv would be cool for looking at plant life. The seek and flir imagers are cool for IR
  - bliteben 5y ago
    UV seems more possible now that I think about it, thermal would present huge issues with export regulations
sho 5y ago
> Lidar! Radar! Refresh rate! A 7th camera! [..] Who the hell uses any of that stuff? Well... you're right about the lidar, but I absolutely care about the camera and refresh rate. As a lifelong photography geek I basically upgrade for the camera improvements every year.. Camera, messaging, maps and dating apps are basically the only things I use my phone for these days...
- technobabbler 5y ago
  That's cool. I'm glad there are still people interested in photography for its own sake instead of posing for Instagram!
- adonese 5y ago
  What do you suggest for a good camera with a tight budget? I used to have Samsung Galaxy S20 FE and it was just amazing. Trying to buy a new android preferably from flagship killers type of phones
- saiya-jin 5y ago
  Lidar is actually really useful for portraits, Samsung S22 ultra uses it extensively and can find a single hair strand it doesn't blur compared to the background. Everything looks more realistic compared to pure software processing where I can spot errors quickly. As frequent user my good old Nikon full frame (D750), I am really astonished by output these tiny cameras produce. Sure, its not for big screen/print unless we're talking about sunny day. But for everything else, it makes fantastic portraits. I got fed up with the need to carry big heavy pouch around all the time, and missed way too many pictures of my kids to rely on it anymore. It also sees in the dark much better than I do with my eyes, also thanx to Lidar - the pics I take from night walks where there is basically no light source hundreds meters around and I am in the forest (yes my night walks take me sometimes to interesting places, I normally don't use any light). Very dark scene becomes full of (usually) true colors and details simply invisible to me. 10x zoom on my phone allows me to read signs not readable to my eyesight (digital 'AI' zoom is very useful up to cca 30x). I was skeptical too, and its true full frame is still so far ahead, but at what cost - bulk, weight, the need to spend hours on postprocessing batch of photos instead of quick edits in phone in few seconds.
  - technobabbler 5y ago
    Yeah, portrait mode is basically face ID in reverse and works a lot better than room scanning. How does the lidar help with night shots though? Maybe focusing, but otherwise isn't it just a long exposure with algorithmic corrections?
amelius 5y ago
Glad they didn't include a laserpointer.
smoldesu 5y ago
> Those gimmicks help to prevent the phones from becoming the next race to the bottom that the PC and cheap Android markets have become. See: it allows them to keep increasing the cost of their devices while simultaneously preventing them from passing the savings from their streamlined manufacturing to the customer.
andybak 5y ago
Why is 3d scanning any more of a gimmick than photography? When I travel I scan things for the same reason I photograph things.
- technobabbler 5y ago
  The lidar plays a trivial role in that. Most of it is just algorithmic photogrammetry. If you turned the laser off you'd probably still get similar results. And it's a gimmick because it's a tiny niche of people who do that, and I wish I didn't have to pay extra for a phone for a lidar. Would take a fingerprint scanner any day. Or a headphone jack.
  - andybak 5y ago
    > Most of it is just algorithmic photogrammetry. If you turned the laser off you'd probably still get similar results. This is simply untrue. Most 3D scanning apps on iOS make no use of photogrammetry or they do it as an alternate mode for smaller objects. I don't know of one 3D scanning app that combines the LIDAR with photogrammetry. They use one or the other depending on the scale of the object you're scanning. > And it's a gimmick because it's a tiny niche of people who do that, It's not the reason Apple added it. They added it for AR and photography. Arguably the former is a gimmick but the latter is future proofing and might end up being a game-changer.

peterburkimsher 5y ago

Could the 3D model be used to interpolate the shape of an animal behind a cage? I'm imagining a panda enclosure, with lots of people taking photos. 2D images are obscured by a line where the cage is. Eyes can correct for it by moving our heads around, but the photos can't. There is an opportunity for cage-removal to be solved with software (guessing, or taking multiple photos and cross-stitching) or simpler hardware (camera on bottom of phone instead of top, use parallax like 2 eyes). But the lidar option might work too.

samvher 5y ago

Why do you think it has so little use? One thing that comes to my mind is portrait background blur - so far computational implementations on smartphones that I’ve seen were not very good, I suspect this can improve effects like that a lot which seems relevant to consumers. On the forward looking side I think app developers can also get a lot of value out of this. Eg (just a thought) changing the way clothes or shoes are bought by letting you scan body parts to determine fit. And this can generate a lot of data feeding into their vision models + what you already mention about AR/VR.

berta 5y ago
Well, it doesn't seem to me that Apple has backed this new technology with any significant app that makes use of it. And I don't see any serious app in the apple store that makes use of lidar either.
- CubsFan1060 5y ago
  I think you dismissed the portrait mode thing, which I think is by far the biggest use. Two other things I love -- "measure". It's very accurate. Also, there are several apps that allow you to scan an area and then send that to people. I've used it to show people an area (my office, a cool brewery I went to, etc). "3d scanner app" is one that I've used. Interestingly, I also used it to scan a piece of furniture I liked, which let me get precise measurements at a later date. Edit: I forgot one super cool use of measure. Let's say you have a basement with an open ceiling. You can use the measure app to, say, measure a pipe. Then, when you walk upstairs, you can still see the pipe in 3D. So you can then understand just exactly under the floor where the pipe is. Very useful for things like pipes and electric wires.
  - berta 5y ago
    The last one is pretty cool indeed. Still, I feel like Apple has still some "one more thing" for lidar in the future
  - Kon-Peki 5y ago

tyingq 5y ago

There's some practical stuff, like mapping a room so that you can see wallpaper or paint choices on your own walls. Or getting clothing sizes more accurately, etc.

deleted 5y ago

[deleted]

nikhizzle 5y ago

Focusing a camera in low light is still not great with traditional methods such as phase detection. LiDAR focus is spot on every time. The quality of the camera is one of the main selling points of the iPhone (Eg. The ad campaign on billboards is “Shot on iPhone”. So I would argue it is a very appropriate feature.

acthrowthrow123 5y ago

I'm iOS dev and few limitation: 1) Biggest one Lidar is only on iPhones Pro in contrast to TrueDepth that after iPhone X landed in all iPhone devices 2) Lidar has very low resolution I wish the added Lidar to all iPhones + combined this with TrueDepth (both in front and back of iphone) - true depth has better resolution but is slower (only 30fps + higher latency) and only works best for distance < 1 meter