4 ms·

I don't know for sure, but things like text recognition (Live Text) or object recognition in Photos (Visual Look Up) are obvious candidates. I think neural eng

I don't know for sure, but things like text recognition (Live Text) or object recognition in Photos (Visual Look Up) are obvious candidates. I think neural engine is absolutely key to Apple's strategy. They want people to buy expensive devices and they don't want to process user data on their servers. Users get privacy. Apple gets money. It's a pretty coherent business model.

flutas 4y ago
> They want people to buy expensive devices and they don't want to process user data on their servers. > Users get privacy. Apple gets money. Apple also gets users to subsidize the cost of compute indefinitely (by buying the expensive phone), rather than using their servers.
- blululu 4y ago
  It’s not a subsidy. It’s a pricing structure for a commercial transaction. Fundamentally a business can not just give out free compute. In the long run the user of computation needs to pay for it. It’s a question of whether people feel more satisfied paying for it in a lump sum bundled with a device or through a subscription plan on the cloud. For frequently, on demand, low latency applications I would suspect that people will always be happier running the computations locally.
- saagarjha 4y ago
  Apple also runs an OS on that device, so they can't just offload infinite computation for it: it would use too much battery.
- GeekyBear 4y ago
  > Apple also gets users to subsidize the cost of compute indefinitely (by buying the expensive phone), rather than using their servers. Subsidizing the smart speaker hardware and running everything on their servers hasn't worked out well for Alexa and Google Assistant. > The Alexa division is part of the "Worldwide Digital" group along with Amazon Prime video, and Business Insider says that division lost $3 billion in just the first quarter of 2022, with "the vast majority" of the losses blamed on Alexa. That is apparently double the losses of any other division, and the report says the hardware team is on pace to lose $10 billion this year. It sounds like Amazon is tired of burning through all that cash. Google expressed basically identical problems with the Google Assistant business model last month. There's an inability to monetize the simple voice commands most consumers actually want to make, and all of Google's attempts to monetize assistants with display ads and company partnerships haven't worked. With the product sucking up server time and being a big money loser, Google responded just like Amazon by cutting resources to the division. https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-colossal-failure-on-pace-to-lose-10-billion-this-year/ https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-co...
jjoonathan 4y ago
Privacy isn't the only benefit of local compute, users also get colossal bandwidth, tiny latency, and high reliability.
- fauigerzigerk 4y ago
  Agreed. On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time because it would otherwise drain the battery. Where low latency is not an absolute necessity, the economic pull of the cloud will be tremendous, especially if mobile networks become ubiquitous and fast.
  - vinay_ys 4y ago
    That's a weak argument. Lots of hardware sits idle in the cloud as well. And on your phone its not expensive. In fact, the $/tflop is cheaper on phone than in the cloud – cloud has to deal with all kinds of complexity that you assume away in your local single-tenant phone context.
    - fauigerzigerk 4y ago
      I wouldn't be so sure. A quick web search brings up average server utilisation numbers for large-scale cloud providers between 45% and 65%. That's probably an order of magnitude or two higher than what you could do on a mobile device without absolutely annihilating the battery.
  - nl 4y ago
    Inactive silicon isn't inefficient. And it's not even particularly wasteful to produce. While there are lots of devices the amount of materials in each chip is incredibly small. In terms of financial cost the material cost is so low that most chip vendors include features in low end chips that are disabled but shipped anyway.
    - moffkalast 4y ago
gcr 4y ago
From my understanding (which seems to differ from OP), the benefit of running models on ANE is for much lower power usage, not necessarily speed. Apple’s always-on Hey Siri wake word detection uses ANE for minimal battery life. I’m not sure but I think android’s “Now Playing” feature (the always-on Shazam thing that shows the names of songs playing around you on your lock screen ) also uses Edge TPU for similar reasons.

flutas 4y ago

> They want people to buy expensive devices and they don't want to process user data on their servers. > Users get privacy. Apple gets money. Apple also gets users to subsidize the cost of compute indefinitely (by buying the expensive phone), rather than using their servers.

blululu 4y ago
It’s not a subsidy. It’s a pricing structure for a commercial transaction. Fundamentally a business can not just give out free compute. In the long run the user of computation needs to pay for it. It’s a question of whether people feel more satisfied paying for it in a lump sum bundled with a device or through a subscription plan on the cloud. For frequently, on demand, low latency applications I would suspect that people will always be happier running the computations locally.
saagarjha 4y ago
Apple also runs an OS on that device, so they can't just offload infinite computation for it: it would use too much battery.
GeekyBear 4y ago
> Apple also gets users to subsidize the cost of compute indefinitely (by buying the expensive phone), rather than using their servers. Subsidizing the smart speaker hardware and running everything on their servers hasn't worked out well for Alexa and Google Assistant. > The Alexa division is part of the "Worldwide Digital" group along with Amazon Prime video, and Business Insider says that division lost $3 billion in just the first quarter of 2022, with "the vast majority" of the losses blamed on Alexa. That is apparently double the losses of any other division, and the report says the hardware team is on pace to lose $10 billion this year. It sounds like Amazon is tired of burning through all that cash. Google expressed basically identical problems with the Google Assistant business model last month. There's an inability to monetize the simple voice commands most consumers actually want to make, and all of Google's attempts to monetize assistants with display ads and company partnerships haven't worked. With the product sucking up server time and being a big money loser, Google responded just like Amazon by cutting resources to the division. https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-colossal-failure-on-pace-to-lose-10-billion-this-year/ https://arstechnica.com/gadgets/2022/11/amazon-alexa-is-a-co...

jjoonathan 4y ago

Privacy isn't the only benefit of local compute, users also get colossal bandwidth, tiny latency, and high reliability.

fauigerzigerk 4y ago
Agreed. On the downside, we have to acknowledge that it is hugely inefficient for everyone to own expensive hardware that has to sit idle most of the time because it would otherwise drain the battery. Where low latency is not an absolute necessity, the economic pull of the cloud will be tremendous, especially if mobile networks become ubiquitous and fast.
- vinay_ys 4y ago
  That's a weak argument. Lots of hardware sits idle in the cloud as well. And on your phone its not expensive. In fact, the $/tflop is cheaper on phone than in the cloud – cloud has to deal with all kinds of complexity that you assume away in your local single-tenant phone context.
  - fauigerzigerk 4y ago
    I wouldn't be so sure. A quick web search brings up average server utilisation numbers for large-scale cloud providers between 45% and 65%. That's probably an order of magnitude or two higher than what you could do on a mobile device without absolutely annihilating the battery.
- nl 4y ago
  Inactive silicon isn't inefficient. And it's not even particularly wasteful to produce. While there are lots of devices the amount of materials in each chip is incredibly small. In terms of financial cost the material cost is so low that most chip vendors include features in low end chips that are disabled but shipped anyway.
  - moffkalast 4y ago

gcr 4y ago

From my understanding (which seems to differ from OP), the benefit of running models on ANE is for much lower power usage, not necessarily speed. Apple’s always-on Hey Siri wake word detection uses ANE for minimal battery life. I’m not sure but I think android’s “Now Playing” feature (the always-on Shazam thing that shows the names of songs playing around you on your lock screen ) also uses Edge TPU for similar reasons.