4 ms·

It’s used for a variety of things: - Biometrics (Face ID and Touch ID) - Image analysis (face matching, aesthetic evaluations, etc) - Text to speech and spee

It’s used for a variety of things: - Biometrics (Face ID and Touch ID) - Image analysis (face matching, aesthetic evaluations, etc) - Text to speech and speech to text (smaller models on device, used for privacy/latency/reliability) - Small ad-hoc models like Raise to Speak on Apple Watch, the Hey Siri detector (https://machinelearning.apple.com/research/hey-siri https://machinelearning.apple.com/research/hey-siri) These things have been in phones for 5 years now and have been used from day one

simonw 4y ago
Right, but do any of those things really need 15 trillion operations for second? Have they been getting noticeably better with upgraded phone models?
- jamiek88 4y ago
  Yes definitely. I could only find a blurry YouTube video of the instruction manual for an old old heater in my house. I paused the video on the bit I needed the guy had zoomed into and was able to copy and paste the text that I could barely read into a notes doc. There’s no one splashy thing just lots of little quality of life improvements.
- blululu 4y ago
  No but the first party users should not consume all the compute on the chip. The bigger the margin the better for the device. The other aspect of this is speed and power consumption (battery life is a top 3 phone feature across pretty much all consumers).
- burnished 4y ago
  I got one recently and generally think the phone is garbage, but the OCR built into pictures is really something else. I took a photo of a label for a barcode when I couldnt see it myself but could get my hand nearby, it was at an odd angle, but when I pressed my finger to the text I was interested in the phone captured it immediately, highlighted it, and I copied it nice as you please.
- conradev 4y ago
  You are looking at the wrong top-line number as the “constraint”. My understanding is that the power (and die area) involved in moving the data around is a far greater bottleneck than the actual matmuls themselves. Your iPhone won’t analyze your photos unless it is plugged in. It’s power intensive primarily due to the amount of data that needs to move to classify faces in a photo. Image files are quite large these days, and even if you downscale them you still need to load them from disk and decompress them in the first place. The scarce resource on the iPhone is RAM, not compute. The Camera’s image pipeline uses so much memory that the phone is effectively unable to multi-task when the camera is open. Newer phones are running on faster process nodes, so that is always a boost, but the phones also have more RAM for bigger cameras. Apple’s unified memory architecture is a key advantage, because they can ship less RAM if it can be shared across the CPU/GPU, but all mobile SoCs have unified memory for this reason, so it’s not Apple’s secret sauce, just part of it. I think the primary constraint will be how many models can fit in memory at once, not how many can run at once. They also don’t often all run at once: you only use the TTS model when Siri speaks, so Siri only loads it when it’s needed. But, in that case, then the latency of loading the model from disk is a concern… In short, performance for ML is really a small subset of performance more generally, and Apple takes performance seriously because efficiency is their game. That is how good battery life pops out on the other side.

simonw 4y ago

Right, but do any of those things really need 15 trillion operations for second? Have they been getting noticeably better with upgraded phone models?

jamiek88 4y ago
Yes definitely. I could only find a blurry YouTube video of the instruction manual for an old old heater in my house. I paused the video on the bit I needed the guy had zoomed into and was able to copy and paste the text that I could barely read into a notes doc. There’s no one splashy thing just lots of little quality of life improvements.
blululu 4y ago
No but the first party users should not consume all the compute on the chip. The bigger the margin the better for the device. The other aspect of this is speed and power consumption (battery life is a top 3 phone feature across pretty much all consumers).
burnished 4y ago
I got one recently and generally think the phone is garbage, but the OCR built into pictures is really something else. I took a photo of a label for a barcode when I couldnt see it myself but could get my hand nearby, it was at an odd angle, but when I pressed my finger to the text I was interested in the phone captured it immediately, highlighted it, and I copied it nice as you please.
conradev 4y ago
You are looking at the wrong top-line number as the “constraint”. My understanding is that the power (and die area) involved in moving the data around is a far greater bottleneck than the actual matmuls themselves. Your iPhone won’t analyze your photos unless it is plugged in. It’s power intensive primarily due to the amount of data that needs to move to classify faces in a photo. Image files are quite large these days, and even if you downscale them you still need to load them from disk and decompress them in the first place. The scarce resource on the iPhone is RAM, not compute. The Camera’s image pipeline uses so much memory that the phone is effectively unable to multi-task when the camera is open. Newer phones are running on faster process nodes, so that is always a boost, but the phones also have more RAM for bigger cameras. Apple’s unified memory architecture is a key advantage, because they can ship less RAM if it can be shared across the CPU/GPU, but all mobile SoCs have unified memory for this reason, so it’s not Apple’s secret sauce, just part of it. I think the primary constraint will be how many models can fit in memory at once, not how many can run at once. They also don’t often all run at once: you only use the TTS model when Siri speaks, so Siri only loads it when it’s needed. But, in that case, then the latency of loading the model from disk is a concern… In short, performance for ML is really a small subset of performance more generally, and Apple takes performance seriously because efficiency is their game. That is how good battery life pops out on the other side.