4 ms·

My speculation is that the small on-device models are simply not useful enough for practical purposes. The on-device transformer models are described as having

My speculation is that the small on-device models are simply not useful enough for practical purposes. The on-device transformer models are described as having 3B parameters[0]. Their own figures from 6 months ago show that humans prefer other models' output 25-40% of the time, which aligns with Gurman's reporting. I don't know how well Apple's A-series chips can handle transformers, but if you play with their on-device diffusion models (Image Playground, Genmoji), you can watch your battery drop before your eyes. And the output of these models is also embarrassingly behind the state of the art. If Apple can make great foundation models for Private Cloud Compute, that's great. But then what's the point of buying a new iPhone for on-device inference capabilities? Secondly, I speculate that allowing a server-side model to query the personal context database is going to be hard to do performantly, requiring several round trips and uploading fragments of the data to the cloud for processing. 0: https://machinelearning.apple.com/research/introducing-apple-foundation-models https://machinelearning.apple.com/research/introducing-apple...

seec 2y ago
Apple's approach makes no sense but they don't have a choice because of all the past marketing nonsense. They need to justify their hardware quality/price which is becoming irrelevant because even the cheapest phone work just fine without lags nowadays. AI will be something to be done on servers for the foreseeable future but Apple put itself in a hole with their privacy nonsense. If instead they had truly invested on Macs and allowed their customers to build their own personal cloud, they would have a business case. Instead, they were extremely greedy and tried to double dip with high hardware price and "service" subscriptions (their low storage devices are made just for that). They are going to pay for this short-term thinking, eventually...

seec 2y ago

Apple's approach makes no sense but they don't have a choice because of all the past marketing nonsense. They need to justify their hardware quality/price which is becoming irrelevant because even the cheapest phone work just fine without lags nowadays. AI will be something to be done on servers for the foreseeable future but Apple put itself in a hole with their privacy nonsense. If instead they had truly invested on Macs and allowed their customers to build their own personal cloud, they would have a business case. Instead, they were extremely greedy and tried to double dip with high hardware price and "service" subscriptions (their low storage devices are made just for that). They are going to pay for this short-term thinking, eventually...