You are looking at the wrong top-line number as the “constraint”. My understanding is that the power (and die area) involved in moving the data around is a far

You are looking at the wrong top-line number as the “constraint”. My understanding is that the power (and die area) involved in moving the data around is a far greater bottleneck than the actual matmuls themselves. Your iPhone won’t analyze your photos unless it is plugged in. It’s power intensive primarily due to the amount of data that needs to move to classify faces in a photo. Image files are quite large these days, and even if you downscale them you still need to load them from disk and decompress them in the first place. The scarce resource on the iPhone is RAM, not compute. The Camera’s image pipeline uses so much memory that the phone is effectively unable to multi-task when the camera is open. Newer phones are running on faster process nodes, so that is always a boost, but the phones also have more RAM for bigger cameras. Apple’s unified memory architecture is a key advantage, because they can ship less RAM if it can be shared across the CPU/GPU, but all mobile SoCs have unified memory for this reason, so it’s not Apple’s secret sauce, just part of it. I think the primary constraint will be how many models can fit in memory at once, not how many can run at once. They also don’t often all run at once: you only use the TTS model when Siri speaks, so Siri only loads it when it’s needed. But, in that case, then the latency of loading the model from disk is a concern… In short, performance for ML is really a small subset of performance more generally, and Apple takes performance seriously because efficiency is their game. That is how good battery life pops out on the other side.