4 ms·

I really hope Apple continues its privacy-first ethos and offers an on-device LLM. It would also be great if they could use Handoff-like technology to let my (a

I really hope Apple continues its privacy-first ethos and offers an on-device LLM. It would also be great if they could use Handoff-like technology to let my (almost always nearby, almost always plugged-in) laptop do the thinking, instead of depleting the battery of my phone. I'd surely buy a bunch of HomePod minis if I could tie them into a local LLM.

croes 3y ago
>privacy-first ethos Apple doesn't have a privacy first ethos, it's more of an All your-data-are-belong-to-us. They just don't like to share the asset of their users data, that's not privacy.
- thefourthchime 3y ago
  That's not true you can have complete end to end encryption. By default. iCloud is not encrypted, but you can encrypt it and nobody but you can get access access your data. It's off by default because if you lose the password then you lose everything and that's a support nightmare.
  - croes 3y ago
    They own the OS they don't need to decrypt the transport if they own the source. They could track every button pressed if they wanted to. https://gizmodo.com/apple-iphone-analytics-tracking-even-when-off-app-store-1849757558 https://gizmodo.com/apple-iphone-analytics-tracking-even-whe...
  - krunck 3y ago
    Who can confirm this though?
    - the_mar 3y ago
      Can and do are two vastly different things. The assumption that data privacy can only be accomplished by no one having any access to data is ridiculous. Of all the companies that exist right now, apple does a fair bit of work regarding data anonymization and access restriction.
- appplication 3y ago
  This is not true at all. I would recommend reading a bit on Apple’s privacy practices and how they differ from other players in the space to arrive at a more informed understanding here. From Apple’s legal page [0] for e.g. maps: “Individual usage metrics are associated with an identifier that rotates multiple times per hour, and is not tied to your Apple ID. This means Maps cannot search for information about you based on an identifier linked to you or your device.” [0]: https://www.apple.com/legal/privacy/data/en/apple-maps/ https://www.apple.com/legal/privacy/data/en/apple-maps/
  - hu3 3y ago
    That's not remotely true, at least not for Apple in China. We know that Apple will throw their users under the bus for profit if incentives align. "Apple tells suppliers to use 'Taiwan, China' or 'Chinese Taipei' to appease Beijing" https://www.theregister.com/2022/08/05/apple_warns_suppliers_in_taiwan/ https://www.theregister.com/2022/08/05/apple_warns_suppliers... "The problem with canceling Jon Stewart: Apple bowed to Chinese government censorship" https://www.usatoday.com/story/opinion/voices/2023/10/26/jon-stewart-apple-tv-show-canceled-china-government-censorship/71298946007/ https://www.usatoday.com/story/opinion/voices/2023/10/26/jon... "Apple pulls Taiwanese flag emoji from iPhones in Hong Kong" https://www.latimes.com/business/story/2019-10-08/apple-taiwan-flag-emoji-iphones-hong-kong https://www.latimes.com/business/story/2019-10-08/apple-taiw... Even Google didn't went that far and withdrew from China. "In early 2010, Google shut down its Chinese website altogether after failing to reach an agreement with Beijing over how much content to censor in search results." https://observer.com/2022/10/alphabet-shut-google-translate-china-big-tech/ https://observer.com/2022/10/alphabet-shut-google-translate-...
checker659 3y ago
I wish they'd put a huge FPGA on every device.
- internetter 3y ago
  For all Apple's flaws, I trust that they could disrupt the AI industry once they enter it. Their decisions are all highly calculated, and they employee vast swaths of researchers. I would not be surprised if they entered the market with a well functioning local LLM
  - mdhb 3y ago
    Have you actually used Siri? They are absolutely garbage at anything even remotely resembling the basics of AI/LLM related after spending a decade on it.
    - internetter 3y ago
      Obviously, they haven't invested any money into it (well, apparently they will for iOS 17). When Apple does invest money, however, which they seem to be doing now, there are usually impressive results. Take the M1, for instance.
      - mdhb 3y ago
        Im not debating the mechanics of why it’s so shit, I’m saying that they are many years behind everyone else and that it’s not obvious that simply throwing money at the problem is going to fix that for them or that they actually have much of a track record of making good software again despite unlimited budgets and many years of opportunity to prove otherwise.
        acchow 3y ago
        Llama2 13b runs on an m1 at 13 tokens/s. So they will be able to do this on an iPhone. https://gist.github.com/gengwg/26592c1979a0bca8b65e2f819e31ab5c https://gist.github.com/gengwg/26592c1979a0bca8b65e2f819e31a...
londons_explore 3y ago
And I hope the reverse. Most of my data I am happy to entrust to Apple, and I don't want them to release substandard not-very-smart products just because they're limited to AI models that only take a few gigs of RAM and can run on a single phone processor. Please just use some huge models to make actually smart products and run them in the cloud if it isn't feasible to run them on-device. Perhaps have an 'offline mode', which runs small models (the google assistant already does that - and it's very noticeable that the online mode has very accurate speech recognition, whereas offline mode can only recognise basic words reliably).
- gnicholas 3y ago
  I assume they'll continue working on Siri-like cloud based AI and will be happy to use that as well.
crooked-v 3y ago
There are lots and lots of rumors suggesting an LLM-driven Siri 2.0, as well as broad integration of minor LLM-driven functionality like locally creating smart playlists.
- Reubachi 3y ago
  I really need to know who thought that the best, first consumer visible iteration of siri 2.0 is to "locally create smart playlists." More impressive would be shortcut automation/smart shortcut tips IE; "Create a shortcut routine based off my device habits for week XYZ." Is there any other examples you've heard talked about?
xp84 3y ago
I am sure Apple leadership agrees with your overall idea of on-device, but don’t see how this could compete with big GPUs in a data center for “actually doing a good job.” Apple is a distant last place in everything to do with AI today, so I’m skeptical that they can suddenly make an impact with both hands tied around their back (in other words 1/100th the TDP, one iPhone chip on battery vs farms of GPUs). I’d honestly more appreciate the ability to choose “best quality with theoretical decrease in ‘privacy’” over “best that you can do without hurting the tiny iPhone battery but perfect privacy.” I’d always pick the first one if given the choice.
- gnicholas 3y ago
  I think the way they could compete is by having on-device handle a limit universe of queries, and then use some souped-up Siri for the rest. Currently iPhones can do some Siri commands even when offline, for example.
  - TaylorAlexander 3y ago
    Hell even the most recent Apple Watch does some Siri commands, including voice to text, on device. Apple is certainly pushing low power on device processing with their own custom chips (S9 on the watch I believe).
    - gnicholas 3y ago
      Yep, I could totally see them enabling this only on Pro-level iPhones, for example. This would presumably be because Pro devices could have different chips that are designed to handle these specific tasks, as well as having larger batteries. I expect there will be a lot of iPhone upgrades this fall, as buyers will finally see a difference between their 1-3 year old device and a new iPhone.
      - TaylorAlexander 3y ago
        Indeed, this is what the rumor mill is suggesting: https://www.macrumors.com/2024/02/14/iphone-16-upgraded-neural-engine-rumor/ https://www.macrumors.com/2024/02/14/iphone-16-upgraded-neur...

croes 3y ago

>privacy-first ethos Apple doesn't have a privacy first ethos, it's more of an All your-data-are-belong-to-us. They just don't like to share the asset of their users data, that's not privacy.

thefourthchime 3y ago
That's not true you can have complete end to end encryption. By default. iCloud is not encrypted, but you can encrypt it and nobody but you can get access access your data. It's off by default because if you lose the password then you lose everything and that's a support nightmare.
- croes 3y ago
  They own the OS they don't need to decrypt the transport if they own the source. They could track every button pressed if they wanted to. https://gizmodo.com/apple-iphone-analytics-tracking-even-when-off-app-store-1849757558 https://gizmodo.com/apple-iphone-analytics-tracking-even-whe...
- krunck 3y ago
  Who can confirm this though?
  - the_mar 3y ago
    Can and do are two vastly different things. The assumption that data privacy can only be accomplished by no one having any access to data is ridiculous. Of all the companies that exist right now, apple does a fair bit of work regarding data anonymization and access restriction.
appplication 3y ago
This is not true at all. I would recommend reading a bit on Apple’s privacy practices and how they differ from other players in the space to arrive at a more informed understanding here. From Apple’s legal page [0] for e.g. maps: “Individual usage metrics are associated with an identifier that rotates multiple times per hour, and is not tied to your Apple ID. This means Maps cannot search for information about you based on an identifier linked to you or your device.” [0]: https://www.apple.com/legal/privacy/data/en/apple-maps/ https://www.apple.com/legal/privacy/data/en/apple-maps/
- hu3 3y ago
  That's not remotely true, at least not for Apple in China. We know that Apple will throw their users under the bus for profit if incentives align. "Apple tells suppliers to use 'Taiwan, China' or 'Chinese Taipei' to appease Beijing" https://www.theregister.com/2022/08/05/apple_warns_suppliers_in_taiwan/ https://www.theregister.com/2022/08/05/apple_warns_suppliers... "The problem with canceling Jon Stewart: Apple bowed to Chinese government censorship" https://www.usatoday.com/story/opinion/voices/2023/10/26/jon-stewart-apple-tv-show-canceled-china-government-censorship/71298946007/ https://www.usatoday.com/story/opinion/voices/2023/10/26/jon... "Apple pulls Taiwanese flag emoji from iPhones in Hong Kong" https://www.latimes.com/business/story/2019-10-08/apple-taiwan-flag-emoji-iphones-hong-kong https://www.latimes.com/business/story/2019-10-08/apple-taiw... Even Google didn't went that far and withdrew from China. "In early 2010, Google shut down its Chinese website altogether after failing to reach an agreement with Beijing over how much content to censor in search results." https://observer.com/2022/10/alphabet-shut-google-translate-china-big-tech/ https://observer.com/2022/10/alphabet-shut-google-translate-...

checker659 3y ago

I wish they'd put a huge FPGA on every device.

internetter 3y ago
For all Apple's flaws, I trust that they could disrupt the AI industry once they enter it. Their decisions are all highly calculated, and they employee vast swaths of researchers. I would not be surprised if they entered the market with a well functioning local LLM
- mdhb 3y ago
  Have you actually used Siri? They are absolutely garbage at anything even remotely resembling the basics of AI/LLM related after spending a decade on it.
  - internetter 3y ago
    Obviously, they haven't invested any money into it (well, apparently they will for iOS 17). When Apple does invest money, however, which they seem to be doing now, there are usually impressive results. Take the M1, for instance.
    - mdhb 3y ago
      Im not debating the mechanics of why it’s so shit, I’m saying that they are many years behind everyone else and that it’s not obvious that simply throwing money at the problem is going to fix that for them or that they actually have much of a track record of making good software again despite unlimited budgets and many years of opportunity to prove otherwise.
      - acchow 3y ago
        Llama2 13b runs on an m1 at 13 tokens/s. So they will be able to do this on an iPhone. https://gist.github.com/gengwg/26592c1979a0bca8b65e2f819e31ab5c https://gist.github.com/gengwg/26592c1979a0bca8b65e2f819e31a...

londons_explore 3y ago

And I hope the reverse. Most of my data I am happy to entrust to Apple, and I don't want them to release substandard not-very-smart products just because they're limited to AI models that only take a few gigs of RAM and can run on a single phone processor. Please just use some huge models to make actually smart products and run them in the cloud if it isn't feasible to run them on-device. Perhaps have an 'offline mode', which runs small models (the google assistant already does that - and it's very noticeable that the online mode has very accurate speech recognition, whereas offline mode can only recognise basic words reliably).

gnicholas 3y ago
I assume they'll continue working on Siri-like cloud based AI and will be happy to use that as well.

crooked-v 3y ago

There are lots and lots of rumors suggesting an LLM-driven Siri 2.0, as well as broad integration of minor LLM-driven functionality like locally creating smart playlists.

Reubachi 3y ago
I really need to know who thought that the best, first consumer visible iteration of siri 2.0 is to "locally create smart playlists." More impressive would be shortcut automation/smart shortcut tips IE; "Create a shortcut routine based off my device habits for week XYZ." Is there any other examples you've heard talked about?

xp84 3y ago

I am sure Apple leadership agrees with your overall idea of on-device, but don’t see how this could compete with big GPUs in a data center for “actually doing a good job.” Apple is a distant last place in everything to do with AI today, so I’m skeptical that they can suddenly make an impact with both hands tied around their back (in other words 1/100th the TDP, one iPhone chip on battery vs farms of GPUs). I’d honestly more appreciate the ability to choose “best quality with theoretical decrease in ‘privacy’” over “best that you can do without hurting the tiny iPhone battery but perfect privacy.” I’d always pick the first one if given the choice.

gnicholas 3y ago
I think the way they could compete is by having on-device handle a limit universe of queries, and then use some souped-up Siri for the rest. Currently iPhones can do some Siri commands even when offline, for example.
- TaylorAlexander 3y ago
  Hell even the most recent Apple Watch does some Siri commands, including voice to text, on device. Apple is certainly pushing low power on device processing with their own custom chips (S9 on the watch I believe).
  - gnicholas 3y ago
    Yep, I could totally see them enabling this only on Pro-level iPhones, for example. This would presumably be because Pro devices could have different chips that are designed to handle these specific tasks, as well as having larger batteries. I expect there will be a lot of iPhone upgrades this fall, as buyers will finally see a difference between their 1-3 year old device and a new iPhone.
    - TaylorAlexander 3y ago
      Indeed, this is what the rumor mill is suggesting: https://www.macrumors.com/2024/02/14/iphone-16-upgraded-neural-engine-rumor/ https://www.macrumors.com/2024/02/14/iphone-16-upgraded-neur...