9 ms·

So the new Apple iPhone X has a front sensor that can generate depth maps, and a rear-sensor that does even more sophisticated depth mapping. They also have a C

So the new Apple iPhone X has a front sensor that can generate depth maps, and a rear-sensor that does even more sophisticated depth mapping. They also have a CPU that is as powerful as anything you get in Desktop PCs. So tell me, what's to keep someone from using the depth information on the front camera, using a Google Dream VR setup (but instead of covering the back of the iphone, use that sensor for the world around them) and then projecting poor man's AR with today's world. If the answer is wrong sensors, they can fix that next year, and do it at a size and a scale that no one else can touch. You just took a hardware problem, and made it a software problem. The fact that Peter Jackson has evidently jumped ship to ARKit is a pretty big sign that Apple is going to get there first. Apple's focus on ML on the phone directly, not requiring a server at the RTT latency is perfect for this application. They have hardware no one can touch, and do full stack miniaturization like no one else. Plus they also attract "lifestyle" buyers.

abritinthebay 9y ago
Tend to agree. The big problem no-one has solved is the glasses/immersive aspect. Hololens is close, but the kit is waaaay to bulky still (and low FoV, not bad tho). In a few years tho... some kind of ARKit accessory will absolutely kill in this space.
Geee 9y ago
Yup. Apple's depth sensor + ML can be also used as pretty powerful 3D hand gesture mapper. Better than Leap Motion I would guess. Also, switching to OLED allows low persistence. Their new custom GPU and custom video encoder on iPhone 8/X might even allow them to do ultra-low-latency AR pipeline. It's coming it's for sure.
knowaveragejoe 9y ago
> They also have a CPU that is as powerful as anything you get in Desktop PCs. Is this actually true?
- dmcy22 9y ago
  Yes it is, at least when using Geekbench as a measure. The A11 beats some of the Intel processors in the 2017 MacBook Pro models. https://www.macrumors.com/2017/09/13/a11-bionic-chip-geekbench-scores/ https://www.macrumors.com/2017/09/13/a11-bionic-chip-geekben...
  - ulber 9y ago
    Well, that makes the CPU as powerful as some processors in mobility focused laptops, not as powerful as "anything you get in desktop PCs".
    - InTheArena 9y ago
      I'm writing this from my linux desktop. It has 11k/3.8k for the geekbench numbers. This is pretty much there.
- valarauca1 9y ago
  No. Geekbenchmark is a bad test. It does a series of short burst tests spaced out over time. These avoid harsh thermal throttling on mobile devices, yet on larger desktops they don't register as a heavy workload so the maximum clock doesn't come into play. Bigger chips will stay down clocked. Effectively what Geekbenchmark's result ends benchmarking sleep state transition speed. Desktop/Laptops don't spin up/down as fast as cellphones so this heavily favors them. Just doing a continuous long test would heavily favor desktops/laptops as they have better designed heatsinks. --- iPhones do have great processors, just fair mobile/desktop test are very hard. They are different chips, that require different power, change clock states in different ways, and have different thermal constraints. The core workloads in most scenarios are very different, so fair for one isn't fair for another.
  - SirHound 9y ago
    I might be talking from ignorance here, but if you put the iPhone chips in a different environment that included the same active cooling that desktop CPUs enjoy, would this still be true?
sitkack 9y ago
By putting AI on the Phone, they invert the data center and communication problem. The mainframe vs smart client cycle continues. A lower powered chip on the client would need a more powerful chip in the datacenter to make up for data transit time. Individual users pay for power, manage heat dissipation and upgrades. This actually saves Apple billions in data center costs while making them money.
- achamayou 9y ago
  And as a nice add-on, they get to brag about being privacy friendly since nothing leaves the device.
  - mercer 9y ago
    I'm becoming quite convinced that long-term this might be Apple's biggest selling-points (and they realize this). In precisely the areas where Apple has or might end up having difficulty competing with the likes of, say, Google and their massive datasets, the focus on privacy and security might very well be enough for consumers to go for the 'inferior' product. For example, I've considered switching away from Apple for my new phone, but the security of the iPhone (or so I've read multiple times here on HN) is enough to keep from even considering a switch to an Android device even if everything else were better on Android. The same goes for my photos. I'd love the features Google Photos offers, but I just don't feel comfortable uploading them all to their servers. While Apple's implementation of these features might be inferior, at least I can keep things on my (secure) devices.
codeonfire 9y ago
Nothing is keeping someone. Probably the depth sensor is not high enough resolution. The camera is not high enough resolution. The phone battery will be dead in 45 minutes. I imagine the high end AR market would be for things like surgery where extreme precision matters. For instance hook a high precision AR device up to a C-arm and the way angioplasty is carried out would be completely changed forever.
- sgift 9y ago
  While AR would be a cool use-case for angioplasty the difference to current methods (https://en.wikipedia.org/wiki/Interventional_radiology https://en.wikipedia.org/wiki/Interventional_radiology) probably isn't as big as you think.
  - codeonfire 9y ago
    I guess I meant data-wise. The image of the catheter and blood vessels would be superimposed in the AR device and the doctor would look at the patient instead of a monitor.
tudorw 9y ago
getting there first is not the same as winning :)
- libertine 9y ago
  Specially when you have players around with unlimited cash.
storgendibal 9y ago
The rear sensor does not do "even more sophisticated depth mapping". TrueDepth sensor is only on the front. The CPU is not as powerful as anything you get in Desktop PCs. There's an order of magnitude power difference between a mobile SoC and a desktop CPU/GPU combination. If you have doubts about that, try running full Photoshop on a mobile SoC via Windows 10 on ARM. Not sure what "Dream VR" is. Do you mean Daydream?

abritinthebay 9y ago

Tend to agree. The big problem no-one has solved is the glasses/immersive aspect. Hololens is close, but the kit is waaaay to bulky still (and low FoV, not bad tho). In a few years tho... some kind of ARKit accessory will absolutely kill in this space.

Geee 9y ago

Yup. Apple's depth sensor + ML can be also used as pretty powerful 3D hand gesture mapper. Better than Leap Motion I would guess. Also, switching to OLED allows low persistence. Their new custom GPU and custom video encoder on iPhone 8/X might even allow them to do ultra-low-latency AR pipeline. It's coming it's for sure.

knowaveragejoe 9y ago

> They also have a CPU that is as powerful as anything you get in Desktop PCs. Is this actually true?

dmcy22 9y ago
Yes it is, at least when using Geekbench as a measure. The A11 beats some of the Intel processors in the 2017 MacBook Pro models. https://www.macrumors.com/2017/09/13/a11-bionic-chip-geekbench-scores/ https://www.macrumors.com/2017/09/13/a11-bionic-chip-geekben...
- ulber 9y ago
  Well, that makes the CPU as powerful as some processors in mobility focused laptops, not as powerful as "anything you get in desktop PCs".
  - InTheArena 9y ago
    I'm writing this from my linux desktop. It has 11k/3.8k for the geekbench numbers. This is pretty much there.
valarauca1 9y ago
No. Geekbenchmark is a bad test. It does a series of short burst tests spaced out over time. These avoid harsh thermal throttling on mobile devices, yet on larger desktops they don't register as a heavy workload so the maximum clock doesn't come into play. Bigger chips will stay down clocked. Effectively what Geekbenchmark's result ends benchmarking sleep state transition speed. Desktop/Laptops don't spin up/down as fast as cellphones so this heavily favors them. Just doing a continuous long test would heavily favor desktops/laptops as they have better designed heatsinks. --- iPhones do have great processors, just fair mobile/desktop test are very hard. They are different chips, that require different power, change clock states in different ways, and have different thermal constraints. The core workloads in most scenarios are very different, so fair for one isn't fair for another.
- SirHound 9y ago
  I might be talking from ignorance here, but if you put the iPhone chips in a different environment that included the same active cooling that desktop CPUs enjoy, would this still be true?

sitkack 9y ago

By putting AI on the Phone, they invert the data center and communication problem. The mainframe vs smart client cycle continues. A lower powered chip on the client would need a more powerful chip in the datacenter to make up for data transit time. Individual users pay for power, manage heat dissipation and upgrades. This actually saves Apple billions in data center costs while making them money.

achamayou 9y ago
And as a nice add-on, they get to brag about being privacy friendly since nothing leaves the device.
- mercer 9y ago
  I'm becoming quite convinced that long-term this might be Apple's biggest selling-points (and they realize this). In precisely the areas where Apple has or might end up having difficulty competing with the likes of, say, Google and their massive datasets, the focus on privacy and security might very well be enough for consumers to go for the 'inferior' product. For example, I've considered switching away from Apple for my new phone, but the security of the iPhone (or so I've read multiple times here on HN) is enough to keep from even considering a switch to an Android device even if everything else were better on Android. The same goes for my photos. I'd love the features Google Photos offers, but I just don't feel comfortable uploading them all to their servers. While Apple's implementation of these features might be inferior, at least I can keep things on my (secure) devices.

codeonfire 9y ago

Nothing is keeping someone. Probably the depth sensor is not high enough resolution. The camera is not high enough resolution. The phone battery will be dead in 45 minutes. I imagine the high end AR market would be for things like surgery where extreme precision matters. For instance hook a high precision AR device up to a C-arm and the way angioplasty is carried out would be completely changed forever.

sgift 9y ago
While AR would be a cool use-case for angioplasty the difference to current methods (https://en.wikipedia.org/wiki/Interventional_radiology https://en.wikipedia.org/wiki/Interventional_radiology) probably isn't as big as you think.
- codeonfire 9y ago
  I guess I meant data-wise. The image of the catheter and blood vessels would be superimposed in the AR device and the doctor would look at the patient instead of a monitor.

tudorw 9y ago

getting there first is not the same as winning :)

libertine 9y ago
Specially when you have players around with unlimited cash.

storgendibal 9y ago

The rear sensor does not do "even more sophisticated depth mapping". TrueDepth sensor is only on the front. The CPU is not as powerful as anything you get in Desktop PCs. There's an order of magnitude power difference between a mobile SoC and a desktop CPU/GPU combination. If you have doubts about that, try running full Photoshop on a mobile SoC via Windows 10 on ARM. Not sure what "Dream VR" is. Do you mean Daydream?