4 ms·

Apple's silicon enables them to run incredibly strong models locally, fully offline. This is a huge win over competition that needs clients to connect to a netw

Apple's silicon enables them to run incredibly strong models locally, fully offline. This is a huge win over competition that needs clients to connect to a network-connected GPU farm. Siri wouldn't need to reach out to the web, it's various functions could be abstracted away and driven by embedded models. They've already started positioning for this. The newest M4 Ipad release has a 10 core GPU/16Gi RAM that could be consumed as VRAM. Could run Llama 3 8Gb Instruct with that and get response content much faster than normal speaking cadence.

lern_too_spel 2y ago
The competition has been running on device models with specialized hardware for longer than Apple has. Even with on-device LLMs, Apple is already far behind. https://store.google.com/intl/en/ideas/articles/pixel-feature-drop-december-2023/ https://store.google.com/intl/en/ideas/articles/pixel-featur...
- Atreiden 2y ago
  Sure they're quite behind in the model space, but there are plenty of routes for them to catch up quickly in this regard. Licensing, open-source models, etc. Google's hardware is just not as powerful. The Tensor G3 is far less capable than the A16 let alone the A17 Pro or M4 chip that powers the new iPad. Gemini Nano is a tiny model with limited applications. On the page you've linked, it seems like it's mainly used for text summarization, autocomplete, and various image features. It's not used for the Google assistant, likely due to the complexity of that task relative to the strength of the model. Apple's hardware could run more powerful models that make assistant a first-class citizen. And as another poster mentioned, Apple does not have the same incentive structure as Google (which always has to keep their ad revenue business in mind). Apple would happily offer an all-in-one, does-everything assistant to their users, if it were good enough to compel sales. Google likely would not, but would rather bake integrations into their other products in order to "augment them with AI" while preserving their revenue model.
- jdminhbg 2y ago
  This isn’t really the competition. This is more akin to opening up the OpenAI app than using Siri. Hey Google is, like Siri, still not LLM-based.
  - lern_too_spel 2y ago
    I'm responding to the following. > Apple's silicon enables them to run incredibly strong models locally, fully offline. This is a huge win over competition that needs clients to connect to a network-connected GPU farm. The competition has already racked up that win.
    - jdminhbg 2y ago
      I stand by my counterpoint: Google can’t run “Hey Google” on-device because it only controls a small % of the devices it’s available on. In the world of built-in voice assistants, Siri is behind, but only as behind as it was in 2021.
      - lern_too_spel 2y ago
        This thread is about the world of on-device models, not the world of built-in voice assistants. The competition that this thread is about comes from this sentence in the original comment: "This is a huge win over competition that needs clients to connect to a network-connected GPU farm."

lern_too_spel 2y ago

The competition has been running on device models with specialized hardware for longer than Apple has. Even with on-device LLMs, Apple is already far behind. https://store.google.com/intl/en/ideas/articles/pixel-feature-drop-december-2023/ https://store.google.com/intl/en/ideas/articles/pixel-featur...

Atreiden 2y ago
Sure they're quite behind in the model space, but there are plenty of routes for them to catch up quickly in this regard. Licensing, open-source models, etc. Google's hardware is just not as powerful. The Tensor G3 is far less capable than the A16 let alone the A17 Pro or M4 chip that powers the new iPad. Gemini Nano is a tiny model with limited applications. On the page you've linked, it seems like it's mainly used for text summarization, autocomplete, and various image features. It's not used for the Google assistant, likely due to the complexity of that task relative to the strength of the model. Apple's hardware could run more powerful models that make assistant a first-class citizen. And as another poster mentioned, Apple does not have the same incentive structure as Google (which always has to keep their ad revenue business in mind). Apple would happily offer an all-in-one, does-everything assistant to their users, if it were good enough to compel sales. Google likely would not, but would rather bake integrations into their other products in order to "augment them with AI" while preserving their revenue model.
jdminhbg 2y ago
This isn’t really the competition. This is more akin to opening up the OpenAI app than using Siri. Hey Google is, like Siri, still not LLM-based.
- lern_too_spel 2y ago
  I'm responding to the following. > Apple's silicon enables them to run incredibly strong models locally, fully offline. This is a huge win over competition that needs clients to connect to a network-connected GPU farm. The competition has already racked up that win.
  - jdminhbg 2y ago
    I stand by my counterpoint: Google can’t run “Hey Google” on-device because it only controls a small % of the devices it’s available on. In the world of built-in voice assistants, Siri is behind, but only as behind as it was in 2021.
    - lern_too_spel 2y ago
      This thread is about the world of on-device models, not the world of built-in voice assistants. The competition that this thread is about comes from this sentence in the original comment: "This is a huge win over competition that needs clients to connect to a network-connected GPU farm."