4 ms·

Apple finalizing deal with OpenAI to bring ChatGPT features to iOS 18

115 points by mgh2 2y ago

zer00eyz 2y ago
> A report on Thursday emphasized that iOS 18’s AI features will be powered (in part) by Apple data centers with Apple Silicon processors. The majority of iOS 18’s AI features, however, will be powered entirely on-device, allowing Apple to tout privacy and speed benefits. I wonder if this is the gpt2 we saw recently: https://news.ycombinator.com/item?id=40201486 https://news.ycombinator.com/item?id=40201486
- mritchie712 2y ago
  Many models are already smart enough for so many interesting use cases, but they're too slow to build compelling experiences. What we (speaking selfishly about https://www.definite.app/ https://www.definite.app/) really need are faster models. I really hope the rumors are true about gpt2 and it's a smaller, faster, gpt4ish level of intelligence model.
MisterBiggs 2y ago
I have a hard time believing this since everything Apple has been releasing in the AI space points towards running models locally.
- dannyw 2y ago
  Few thoughts here: 1. You can't fit a XXX billion param model on an iPhone. It's just impossible. A mixed approach where most ML is local; but some are cloud, fits with their existing Siri approach. 2. OpenAI won't share model weights, but if there's one company they'd make an exception for, it'd be Apple. Certainly not their big models, but perhaps they have a distilled, SOTA 3-7B model that they're happy to license, especially with Apple's Secure Enclave and ML model encryption.
  - MisterBiggs 2y ago
    For 1. Apple is certainly trying to fit models onto an iPhone with OpenELM and I think the M4 debuting on an iPad is also posturing for local compute as well. A mixed approach is a great point though and I think I could mostly be happy with that.
    - ffgjgf1 2y ago
      > M4 debuting on an iPad is also posturing for local compute as well. Wouldn’t memory be a much bigger bottleneck? You don’t really need a particularly fast a CPU/GPU to run most basic models that couldn’t even fit into the amount of memory that Apple is offering on their devices (especially if you still want to run other apps).
    - lolinder 2y ago
      > For 1. Apple is certainly trying to fit models onto an iPhone with OpenELM and I think the M4 debuting on an iPad is also posturing for local compute as well. You misunderstand point 1. Three-digit billion parameter models like GPT-4 (or even 3.5) aren't going to fit on modern iPhones, not even close. Even Llama 70b requires 35GB of RAM at q4 quantization, and that's just for the model. Compare that to the iPhone 15's 6GB and you'll see the problem. Apple isn't about to announce an iPhone with 6x as much RAM as any previous model. I'd be shocked if they even doubled it. Their local inference models are going to be tiny and limited, which is fine, but that means they have to go to the cloud to provide all the features people are expecting.
  - a_wild_dandan 2y ago
    You can fit a Q5 Llama-3-8b or many other models on new iPhones.
    - ffgjgf1 2y ago
      It’s not very good though and with 6 GB of memory iPhones would struggle anything else alongside it so forget multitasking or even running the full OS while it’s loaded.. Almost all Apple devices (including most Macs) have very low amounts of memory so Apple hadn’t really positioned themselves that well if their goal is running LLMs locally.
- Tagbert 2y ago
  I suspect that they will be running AI locally but also calling out to server-side services where necessary. I don't expect a ChatGPT integration to be the whole of what Apple has planned for AI on their devices.
- capybara_2020 2y ago
  This is what it says in the article > The majority of iOS 18’s AI features, however, will be powered entirely on-device, allowing Apple to tout privacy and speed benefits.
- Xenoamorphous 2y ago
  I see a lot of talk in HN about how Apple running models on the device is such a great idea, for privacy reasons. However I feel like outside of a few niches like HN, people don’t give a damn about privacy (for example, the company I work for has a website with millions of daily unique visitors; GDPR consent rates are usually around 98%). So if other phones use models running on the cloud and they’re higher quality than the local running ones in iPhones (which will prob be smaller) that might be more important to the average user.
  - jwells89 2y ago
    A big differentiator is that local models have access to more user data without sending anything over the wire (which ignoring privacy implications, isn’t free). Local models can also more easily plug into existing macOS/iOS facilities like AppleScript, Automator, Services, Shortcuts, and NSUserActivity which means they’ll be a lot more functional, with the ability to wire together apps that haven’t explicitly added AI support. In other words there’s a lot of potential here. If implemented well, LLM-powered Siri would have vastly more context and integration than anything else right out of the gate.
  - kfarr 2y ago
    They’re not mutually exclusive. Local model can provide immediate answer with medium quality, remote can provide higher quality but longer round trip But anything is better than old school Siri
  - whatindaheck 2y ago
    Apple is primarily a hardware company. Privacy claims aside, on-device gives Apple a reason to keep making, improving, and selling hardware. I also don’t think the GDPR consent is a good measure. Those suck to opt out of manually and I doubt most people consider finding an extension. Apple sells its privacy as zero to low configuration. You “buy” privacy, you don’t configure it. Whether their claims are misleading or not. That’s extremely appealing. A few other thoughts: * Local models would be a requirement if Advanced Data Protection[0] were enabled * Local models could potentially do some interesting work with hefty data (something media related maybe) that isn’t fit for cloud workflows [0]: https://support.apple.com/guide/security/advanced-data-protection-for-icloud-sec973254c5f/web https://support.apple.com/guide/security/advanced-data-prote...
- sillywalk 2y ago
  It looks like both, and they're apparently using Apple Silicon in their own datacenters. From Bloomberg[0] (archived)[1]: "Apple Inc. will deliver some of its upcoming artificial intelligence features this year via data centers equipped with its own in-house processors, part of a sweeping effort to infuse its devices with AI capabilities. The company is placing high-end chips — similar to ones it designed for the Mac — in cloud-computing servers designed to process the most advanced AI tasks coming to Apple devices, according to people familiar with the matter. Simpler AI-related features will be processed directly on iPhones, iPads and Macs ... The first AI server chips will be the M2 Ultra, which was launched last year as part of the Mac Pro and Mac Studio computers, though the company is already eyeing future versions based on the M4 chip..." ... Relatively simple AI tasks — like providing users a summary of their missed iPhone notifications or incoming text messages — could be handled by the chips inside of Apple devices. More complicated jobs, such as generating images or summarizing lengthy news articles and creating long-form responses in emails, would likely require the cloud-based approach — as would an upgraded version of Apple’s Siri voice assistant. ... For now, Apple is planning to use its own data centers to operate the cloud features, but it will eventually rely on outside facilities — as it does with iCloud and other services. The Wall Street Journal reported earlier on some aspects of the server plan." --- And from [2], Apple has been working on Apple Chips in Data Center (ACDC) for few years, focusing on AI inference, not training. [0] https://www.bloomberg.com/news/articles/2024-05-09/apple-to-power-ios-18-ai-features-with-in-house-server-mac-chips-this-year https://www.bloomberg.com/news/articles/2024-05-09/apple-to-... [1] https://archive.ph/MUswk https://archive.ph/MUswk [2] https://www.medianama.com/2024/05/223-apple-launches-project-develop-ai-chips-data-centers/ https://www.medianama.com/2024/05/223-apple-launches-project...
- ein0p 2y ago
  Por que no los dos? A large LLM won’t fit, but you don’t need a large LLM to invoke functions or answer texts. And when you need chat or search, that can go to the server, same as Siri does today.
- gnicholas 2y ago
  It would be really great if they could respond immediately using locally-generated information, and then seamlessly weave in data that comes back from the cloud.
photochemsyn 2y ago
I really would like to talk to a completely unconstrained LLM running at the upper end of the current technological limit. Not going to tell you what we would talk about.
- leobg 2y ago
  You can. Simply use the base models. You can do anything once you ditch the RLHF and the instruction tuning. Then, even those 7b versions are mindblowingly good. But you’ll need to either change your prompting style or do your own fine-tuning to get the behavior you want.
xyst 2y ago
Siri is about to be put out to pasture. Whoever is working on the Siri team will become the next “Apple Car”.
- JofArnold 2y ago
  Ironically (with respect to what you just wrote) Apple Car team was moved to AI.
Humphrey 2y ago
Due to time constraints, it sounds like this would be a relatively simple integration that operates similar to standard ChatGPT - whereas, the rest of us are all hoping for LLM powered Siri 2.0. On device would be amazing, but I suspect GPT 4.0 would still be too large to be able to run on device - regardless on how much more Neural Engine grunt the next A series Apple chip has.
- Tagbert 2y ago
  I think that server-side AI is just one component in a module approach with them running as much as they can on-device and with their own models. Imagine an AI powered Siri that runs on device to handle a lot of the basic interaction and functions and then calls out to the server for more heavy lifting.
behnamoh 2y ago
The biggest lesson of the last two years is this: There's so much INERTIA in this industry, way more inertia than what most people believed. Let me explain: In the good old GPT-3 era, we already had iOS Shortcuts that could receive voice commands from the user, send it to GPT-3 API, and show/speak the results. I had it on my Apple Watch. It was just one tap away. This made me and many enthusiasts optimistic about receiving official support of such features on the iPhone and Apple Watch. Fast forward two years later, and the AI penetration landscape doesn't look good. Apple just started thinking about "doing something" about "this AI thing" (which they don't call AI btw, but that's Apple being Apple). Google has a redundant Gemini app on stock Android which overlaps with Google Assistant capabilities (so Google Assistant is not actually equipped with large LMs yet), Alexa is still the same Alexa, and Microsoft Copilot is half-baked in terms of UI/UX (I still don't know why Microsoft can't build decent user apps). The lesson is clear: There is so much inertia in this industry that our jobs will be safe for the foreseeable future. Everyone was afraid of AI-generated content polluting the web. Well, here we are, and aside from some ridiculously obvious content which is generated by AI, the web is still the same web we knew in 2022. It's ridiculous how long ordinary users have to wait for this AI stuff to become officially available on their handheld devices by these companies. Like I said, I and many people were enjoying LLMs on iDevices way before OpenAI introduced ChatGPT and its app companion, and way way before Apple even introduces its own version. Speaking of Apple's LLM features, do you think they will be bundled with the rest of Apple One service package? Business-wise, it makes sense for Apple to charge people for iLLM/Siri Pro/ or whatever they call it.
- kredd 2y ago
  I agree about the jobs, but not about the web part. Any image or blob of text have absolutely lost its meaning. Every comment has an undertone of “is it written by a human?” question surrounding it. It wasn’t really the case 5 years ago, and as it became ubiquitous, even my friends who are not in tech-adjacent fields feel the same way. Benefit of the doubt that one is a human behind a comment has been shifted, so now you have to prove you’re not actually an LLM-bot. Unfortunately it’s only going to get worse. Just like how digital art has zero significance nowadays (hay-days of 2008 deviantART, when some digital art was considered impressive, is gone), so will a lot of other things. My only hope is, it will boost something else that we don’t know yet, or maybe we’ll go back to prioritizing the reality.
Tagbert 2y ago
It looks like Apple will have a modular approach to AI. With some third-party components and some their own. They have and will double down on AI within apps, like selecting the subject in photos, transformer driven typing suggestions, and other app features. They will have a way to call out to a third-party, server-side GPT-style AI like ChatGPT for some functions. They will have one more more local AIs running on-device and possibly replacing what Siri does now and tying the other AI together. This would likely intercept the user's request, infer the meaning, and match that to one or more service to satisfy that request. This is probably like what Rabbit R1 wanted to be but didn't have access to the features that Apple AI would. Apple has been building the components of this for years. Their App Intents interface lets apps publish their features and make them available to call from automation like Shortcuts or an LLM powered Siri. They have been shipping devices with neural processors for 7 years. They have been publishing papers about ways to run LLMs and other AI on device and with reduced resources. If you aren't building a web answerbot yourself, you don't need the full scope of a ChatGPT locally.
- deleted 2y ago
  [deleted]
- choppaface 2y ago
  I like this prediction. Apple Maps version 1 was a huge flop and Siri’s capabilities have been quite conservative despite their investment, and despite Apple getting caught recording users to try to improve it. It makes a lot of sense for Apple to deploy hardware and small on-device LLMs as extremely overpowered “routing” modules that do basic things locally and then farm out other things to a cloud partner. This also fits in well with OpenAI allegedly launching some sort of assistant Monday, so then Apple can point to it if it does well (and integrate it) or at the last minute fall back to Google if OpenAI flops. I wonder if Tim Apple models the Google search deal ending in the next 2-3years, and if there’s a contingency in case chat-based interfaces flop. Given how the car got canned, it would be surprising to see Apple try to make their own GPT-5 / search engine, so the 5-7 year plan must be to remain largely dependent on Google and Microsoft (and to keep Facebook out of the party).
- shepherdjerred 2y ago
blackeyeblitzar 2y ago
It’s a little disappointing that the way most people will experience AI is going to be through just a few companies (Apple, Microsoft, OpenAI). If they all adopt a similar approach to censorship (safety), that creates various problems for society. Not to mention that kind of economic and cultural power should not be in the hands of a few.
- surfpel 2y ago
  That’s also just the general state of technology. Mass market services are highly consolidated and variety exists at the fringes.
mvdtnz 2y ago
I use many products with AI features bolted on, and I haven't used a single one beyond briefly playing to see what it can do (usually very little). Is this really what we're spending billions of dollars on?
- cjk2 2y ago
  If they buy anything they are not confident in externally they can get rid of it quickly.
- DeathArrow 2y ago
  > Is this really what we're spending billions of dollars on? We aren't. It's companies that are spending the money. They want us to buy their products and they are afraid of missing out. It's a new gold rush.
- dclowd9901 2y ago
  If there’s anywhere I could use good LLM, it’s on my phone and in my IDE.
DeathArrow 2y ago
Why aren't they using open source models like Llama3?
- anonylizard 2y ago
  Like have you checked the license of Llama3? Who told you it was free for all commercial use? Most open source licenses by big companies these days, are anti-big-tech. Aka anyone can use them but big tech. "If, on the Meta Llama 3 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights."
  - DeathArrow 2y ago
    Thanks, that explains it. I didn't read the license.
  - asadotzler 2y ago
    Those are NOT open source licenses. Stop calling them that and we'll have fewer people like the op confused.
ok_dad 2y ago
I really don’t want this in my phone unless I choose to have it, and right now I’m pretty upset that Siri can’t be fully disabled in one switch, but I’ll really be much more upset if I have to have ‘Open’AI forced into my life. I truly don’t know what to do today, other than not having a phone, in order to get privacy and a phone that works without any BS.
- bamboozled 2y ago
  I immediately thought the same, I don’t want it. I personally don’t find ChatGPT very useful. Let’s just hope it’s opt out.
  - hackernewds 2y ago
    Curious why not?
    - ghaff 2y ago
      Not the parent but I just don’t. I have used it to basically help with some largely boilerplate writing—as in get a pre-draft down on a page and adapt it from there. Don’t do a lot of programming any longer. It’s sometimes useful but at this point it just isn’t revolutionary for anything I do.
  - tropicalbeach 2y ago
    People like this live in the stone age
    - wiseowise 2y ago
      >> I don’t want to use half-baked, heavily censored, hallucinating piece of crap that will affect battery life of my device > you’re living in the Stone Age, lolz
    - weikju 2y ago
      Your comment isn’t helpful to have a good and useful conversation about this. There are multiple viewpoints and levels of acceptance of new technology. Skepticism is healthy and even warranted. We don’t have to gleefully embrace every new shiny thing the tech companies throw at us. Heck the founder of this site himself once wrote an essay cautioning about adopting new services or tech too quickly because it is too often turned against us [1]: > It took a while though—on the order of 100 years. And unless the rate at which social antibodies evolve can increase to match the accelerating rate at which technological progress throws off new addictions, we'll be increasingly unable to rely on customs to protect us. [3] Unless we want to be canaries in the coal mine of each new addiction—the people whose sad example becomes a lesson to future generations—we'll have to figure out for ourselves what to avoid and how. It will actually become a reasonable strategy (or a more reasonable strategy) to suspect everything new. [1] https://paulgraham.com/addiction.html https://paulgraham.com/addiction.html
Apocryphon 2y ago
This is going to severely undercut Rabbit's "Large Action Model" shtick, isn't it
- ENGNR 2y ago
  Rabbits main play is really to bypass the App Store using the web interface for everything. Otherwise it would have just been an app.
userbinator 2y ago
It doesn't surprise me after seeing that Apple just did this: https://news.ycombinator.com/item?id=40313733 https://news.ycombinator.com/item?id=40313733 Also, is it just me or does the example of asking ChatGPT for gift ideas seem rather "dehumanising", for lack of a better term?
- shepherdjerred 2y ago
  How is this deal related to the iPad ad you linked?
devinprater 2y ago
I can't wait to see what accessibility updates they can do with local/remote AI! I'm definitely getting the Pro iPhone this year if it's running a faster CPU or has more RAM for bigger models. And that could be a selling point for the pro phones this year. Get AppleLLM 3B on a 6 GB iPhone, or Apple 7B which can describe images/videos right on device, describe animations in apps, and try to describe what's happening in all those emulated video games, with the 8/12GB iPhone Pro. I know, wild. But Apple's current screen recognition, using image classification ML from like 3 years ago, works a good 1 description per 2 seconds or so. If they can make AI updates that fast, with a buffer to only keep describing changes in the scene/video, I think it'd work well. On the other hand, isn't it a bit late for Apple to be finalizing things before WWDC? I mean, that doesn't give them that much time to make the video and prepare dev builds. And then they have to tune the system prompt, slide it into the OS (although maybe they have just an API field they can just plug it into), and then do QA. I don't know. Maybe this is old info. Still, I'll be eagerly awaiting any VoiceOver/accessibility updates!
osigurdson 2y ago
I used Siri for a while, then laughed how bad it was compared to Google, then turned it off (also unplugged Google home spying thing and Amazon spying thing, now in same box as Wii). Maybe this will turn things around, but I probably will never turn Siri on again as it will always been at best equally as good as ChatGPT, so why bother?
deleted 2y ago
[deleted]
r0fl 2y ago
Sam Altman on the all in podcast this week Such weird timing to do an interview and avoid on device versions for Apple If this Bloomberg rumour is true, why wouldn’t he schedule that interview for next week? What a waste of everyone’s time
ChrisArchitect 2y ago
Related: Apple Will Revamp Siri to Catch Up to Its Chatbot Competitors https://news.ycombinator.com/item?id=40322104 https://news.ycombinator.com/item?id=40322104

115 points by mgh2 2y ago

> A report on Thursday emphasized that iOS 18’s AI features will be powered (in part) by Apple data centers with Apple Silicon processors. The majority of iOS 18’s AI features, however, will be powered entirely on-device, allowing Apple to tout privacy and speed benefits. I wonder if this is the gpt2 we saw recently: https://news.ycombinator.com/item?id=40201486 https://news.ycombinator.com/item?id=40201486

mritchie712 2y ago
Many models are already smart enough for so many interesting use cases, but they're too slow to build compelling experiences. What we (speaking selfishly about https://www.definite.app/ https://www.definite.app/) really need are faster models. I really hope the rumors are true about gpt2 and it's a smaller, faster, gpt4ish level of intelligence model.

MisterBiggs 2y ago

I have a hard time believing this since everything Apple has been releasing in the AI space points towards running models locally.

dannyw 2y ago
Few thoughts here: 1. You can't fit a XXX billion param model on an iPhone. It's just impossible. A mixed approach where most ML is local; but some are cloud, fits with their existing Siri approach. 2. OpenAI won't share model weights, but if there's one company they'd make an exception for, it'd be Apple. Certainly not their big models, but perhaps they have a distilled, SOTA 3-7B model that they're happy to license, especially with Apple's Secure Enclave and ML model encryption.
- MisterBiggs 2y ago
  For 1. Apple is certainly trying to fit models onto an iPhone with OpenELM and I think the M4 debuting on an iPad is also posturing for local compute as well. A mixed approach is a great point though and I think I could mostly be happy with that.
  - ffgjgf1 2y ago
    > M4 debuting on an iPad is also posturing for local compute as well. Wouldn’t memory be a much bigger bottleneck? You don’t really need a particularly fast a CPU/GPU to run most basic models that couldn’t even fit into the amount of memory that Apple is offering on their devices (especially if you still want to run other apps).
  - lolinder 2y ago
    > For 1. Apple is certainly trying to fit models onto an iPhone with OpenELM and I think the M4 debuting on an iPad is also posturing for local compute as well. You misunderstand point 1. Three-digit billion parameter models like GPT-4 (or even 3.5) aren't going to fit on modern iPhones, not even close. Even Llama 70b requires 35GB of RAM at q4 quantization, and that's just for the model. Compare that to the iPhone 15's 6GB and you'll see the problem. Apple isn't about to announce an iPhone with 6x as much RAM as any previous model. I'd be shocked if they even doubled it. Their local inference models are going to be tiny and limited, which is fine, but that means they have to go to the cloud to provide all the features people are expecting.
- a_wild_dandan 2y ago
  You can fit a Q5 Llama-3-8b or many other models on new iPhones.
  - ffgjgf1 2y ago
    It’s not very good though and with 6 GB of memory iPhones would struggle anything else alongside it so forget multitasking or even running the full OS while it’s loaded.. Almost all Apple devices (including most Macs) have very low amounts of memory so Apple hadn’t really positioned themselves that well if their goal is running LLMs locally.
Tagbert 2y ago
I suspect that they will be running AI locally but also calling out to server-side services where necessary. I don't expect a ChatGPT integration to be the whole of what Apple has planned for AI on their devices.
capybara_2020 2y ago
This is what it says in the article > The majority of iOS 18’s AI features, however, will be powered entirely on-device, allowing Apple to tout privacy and speed benefits.
Xenoamorphous 2y ago
I see a lot of talk in HN about how Apple running models on the device is such a great idea, for privacy reasons. However I feel like outside of a few niches like HN, people don’t give a damn about privacy (for example, the company I work for has a website with millions of daily unique visitors; GDPR consent rates are usually around 98%). So if other phones use models running on the cloud and they’re higher quality than the local running ones in iPhones (which will prob be smaller) that might be more important to the average user.
- jwells89 2y ago
  A big differentiator is that local models have access to more user data without sending anything over the wire (which ignoring privacy implications, isn’t free). Local models can also more easily plug into existing macOS/iOS facilities like AppleScript, Automator, Services, Shortcuts, and NSUserActivity which means they’ll be a lot more functional, with the ability to wire together apps that haven’t explicitly added AI support. In other words there’s a lot of potential here. If implemented well, LLM-powered Siri would have vastly more context and integration than anything else right out of the gate.
- kfarr 2y ago
  They’re not mutually exclusive. Local model can provide immediate answer with medium quality, remote can provide higher quality but longer round trip But anything is better than old school Siri
- whatindaheck 2y ago
  Apple is primarily a hardware company. Privacy claims aside, on-device gives Apple a reason to keep making, improving, and selling hardware. I also don’t think the GDPR consent is a good measure. Those suck to opt out of manually and I doubt most people consider finding an extension. Apple sells its privacy as zero to low configuration. You “buy” privacy, you don’t configure it. Whether their claims are misleading or not. That’s extremely appealing. A few other thoughts: * Local models would be a requirement if Advanced Data Protection[0] were enabled * Local models could potentially do some interesting work with hefty data (something media related maybe) that isn’t fit for cloud workflows [0]: https://support.apple.com/guide/security/advanced-data-protection-for-icloud-sec973254c5f/web https://support.apple.com/guide/security/advanced-data-prote...
sillywalk 2y ago
It looks like both, and they're apparently using Apple Silicon in their own datacenters. From Bloomberg[0] (archived)[1]: "Apple Inc. will deliver some of its upcoming artificial intelligence features this year via data centers equipped with its own in-house processors, part of a sweeping effort to infuse its devices with AI capabilities. The company is placing high-end chips — similar to ones it designed for the Mac — in cloud-computing servers designed to process the most advanced AI tasks coming to Apple devices, according to people familiar with the matter. Simpler AI-related features will be processed directly on iPhones, iPads and Macs ... The first AI server chips will be the M2 Ultra, which was launched last year as part of the Mac Pro and Mac Studio computers, though the company is already eyeing future versions based on the M4 chip..." ... Relatively simple AI tasks — like providing users a summary of their missed iPhone notifications or incoming text messages — could be handled by the chips inside of Apple devices. More complicated jobs, such as generating images or summarizing lengthy news articles and creating long-form responses in emails, would likely require the cloud-based approach — as would an upgraded version of Apple’s Siri voice assistant. ... For now, Apple is planning to use its own data centers to operate the cloud features, but it will eventually rely on outside facilities — as it does with iCloud and other services. The Wall Street Journal reported earlier on some aspects of the server plan." --- And from [2], Apple has been working on Apple Chips in Data Center (ACDC) for few years, focusing on AI inference, not training. [0] https://www.bloomberg.com/news/articles/2024-05-09/apple-to-power-ios-18-ai-features-with-in-house-server-mac-chips-this-year https://www.bloomberg.com/news/articles/2024-05-09/apple-to-... [1] https://archive.ph/MUswk https://archive.ph/MUswk [2] https://www.medianama.com/2024/05/223-apple-launches-project-develop-ai-chips-data-centers/ https://www.medianama.com/2024/05/223-apple-launches-project...
ein0p 2y ago
Por que no los dos? A large LLM won’t fit, but you don’t need a large LLM to invoke functions or answer texts. And when you need chat or search, that can go to the server, same as Siri does today.
gnicholas 2y ago
It would be really great if they could respond immediately using locally-generated information, and then seamlessly weave in data that comes back from the cloud.

photochemsyn 2y ago

I really would like to talk to a completely unconstrained LLM running at the upper end of the current technological limit. Not going to tell you what we would talk about.

leobg 2y ago
You can. Simply use the base models. You can do anything once you ditch the RLHF and the instruction tuning. Then, even those 7b versions are mindblowingly good. But you’ll need to either change your prompting style or do your own fine-tuning to get the behavior you want.

xyst 2y ago

Siri is about to be put out to pasture. Whoever is working on the Siri team will become the next “Apple Car”.

JofArnold 2y ago
Ironically (with respect to what you just wrote) Apple Car team was moved to AI.

Humphrey 2y ago

Due to time constraints, it sounds like this would be a relatively simple integration that operates similar to standard ChatGPT - whereas, the rest of us are all hoping for LLM powered Siri 2.0. On device would be amazing, but I suspect GPT 4.0 would still be too large to be able to run on device - regardless on how much more Neural Engine grunt the next A series Apple chip has.

Tagbert 2y ago
I think that server-side AI is just one component in a module approach with them running as much as they can on-device and with their own models. Imagine an AI powered Siri that runs on device to handle a lot of the basic interaction and functions and then calls out to the server for more heavy lifting.

behnamoh 2y ago

The biggest lesson of the last two years is this: There's so much INERTIA in this industry, way more inertia than what most people believed. Let me explain: In the good old GPT-3 era, we already had iOS Shortcuts that could receive voice commands from the user, send it to GPT-3 API, and show/speak the results. I had it on my Apple Watch. It was just one tap away. This made me and many enthusiasts optimistic about receiving official support of such features on the iPhone and Apple Watch. Fast forward two years later, and the AI penetration landscape doesn't look good. Apple just started thinking about "doing something" about "this AI thing" (which they don't call AI btw, but that's Apple being Apple). Google has a redundant Gemini app on stock Android which overlaps with Google Assistant capabilities (so Google Assistant is not actually equipped with large LMs yet), Alexa is still the same Alexa, and Microsoft Copilot is half-baked in terms of UI/UX (I still don't know why Microsoft can't build decent user apps). The lesson is clear: There is so much inertia in this industry that our jobs will be safe for the foreseeable future. Everyone was afraid of AI-generated content polluting the web. Well, here we are, and aside from some ridiculously obvious content which is generated by AI, the web is still the same web we knew in 2022. It's ridiculous how long ordinary users have to wait for this AI stuff to become officially available on their handheld devices by these companies. Like I said, I and many people were enjoying LLMs on iDevices way before OpenAI introduced ChatGPT and its app companion, and way way before Apple even introduces its own version. Speaking of Apple's LLM features, do you think they will be bundled with the rest of Apple One service package? Business-wise, it makes sense for Apple to charge people for iLLM/Siri Pro/ or whatever they call it.

kredd 2y ago
I agree about the jobs, but not about the web part. Any image or blob of text have absolutely lost its meaning. Every comment has an undertone of “is it written by a human?” question surrounding it. It wasn’t really the case 5 years ago, and as it became ubiquitous, even my friends who are not in tech-adjacent fields feel the same way. Benefit of the doubt that one is a human behind a comment has been shifted, so now you have to prove you’re not actually an LLM-bot. Unfortunately it’s only going to get worse. Just like how digital art has zero significance nowadays (hay-days of 2008 deviantART, when some digital art was considered impressive, is gone), so will a lot of other things. My only hope is, it will boost something else that we don’t know yet, or maybe we’ll go back to prioritizing the reality.

Tagbert 2y ago

It looks like Apple will have a modular approach to AI. With some third-party components and some their own. They have and will double down on AI within apps, like selecting the subject in photos, transformer driven typing suggestions, and other app features. They will have a way to call out to a third-party, server-side GPT-style AI like ChatGPT for some functions. They will have one more more local AIs running on-device and possibly replacing what Siri does now and tying the other AI together. This would likely intercept the user's request, infer the meaning, and match that to one or more service to satisfy that request. This is probably like what Rabbit R1 wanted to be but didn't have access to the features that Apple AI would. Apple has been building the components of this for years. Their App Intents interface lets apps publish their features and make them available to call from automation like Shortcuts or an LLM powered Siri. They have been shipping devices with neural processors for 7 years. They have been publishing papers about ways to run LLMs and other AI on device and with reduced resources. If you aren't building a web answerbot yourself, you don't need the full scope of a ChatGPT locally.

deleted 2y ago
[deleted]
choppaface 2y ago
I like this prediction. Apple Maps version 1 was a huge flop and Siri’s capabilities have been quite conservative despite their investment, and despite Apple getting caught recording users to try to improve it. It makes a lot of sense for Apple to deploy hardware and small on-device LLMs as extremely overpowered “routing” modules that do basic things locally and then farm out other things to a cloud partner. This also fits in well with OpenAI allegedly launching some sort of assistant Monday, so then Apple can point to it if it does well (and integrate it) or at the last minute fall back to Google if OpenAI flops. I wonder if Tim Apple models the Google search deal ending in the next 2-3years, and if there’s a contingency in case chat-based interfaces flop. Given how the car got canned, it would be surprising to see Apple try to make their own GPT-5 / search engine, so the 5-7 year plan must be to remain largely dependent on Google and Microsoft (and to keep Facebook out of the party).
shepherdjerred 2y ago

blackeyeblitzar 2y ago

It’s a little disappointing that the way most people will experience AI is going to be through just a few companies (Apple, Microsoft, OpenAI). If they all adopt a similar approach to censorship (safety), that creates various problems for society. Not to mention that kind of economic and cultural power should not be in the hands of a few.

surfpel 2y ago
That’s also just the general state of technology. Mass market services are highly consolidated and variety exists at the fringes.

mvdtnz 2y ago

I use many products with AI features bolted on, and I haven't used a single one beyond briefly playing to see what it can do (usually very little). Is this really what we're spending billions of dollars on?

cjk2 2y ago
If they buy anything they are not confident in externally they can get rid of it quickly.
DeathArrow 2y ago
> Is this really what we're spending billions of dollars on? We aren't. It's companies that are spending the money. They want us to buy their products and they are afraid of missing out. It's a new gold rush.
dclowd9901 2y ago
If there’s anywhere I could use good LLM, it’s on my phone and in my IDE.

DeathArrow 2y ago

Why aren't they using open source models like Llama3?

anonylizard 2y ago
Like have you checked the license of Llama3? Who told you it was free for all commercial use? Most open source licenses by big companies these days, are anti-big-tech. Aka anyone can use them but big tech. "If, on the Meta Llama 3 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee’s affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights."
- DeathArrow 2y ago
  Thanks, that explains it. I didn't read the license.
- asadotzler 2y ago
  Those are NOT open source licenses. Stop calling them that and we'll have fewer people like the op confused.

ok_dad 2y ago

I really don’t want this in my phone unless I choose to have it, and right now I’m pretty upset that Siri can’t be fully disabled in one switch, but I’ll really be much more upset if I have to have ‘Open’AI forced into my life. I truly don’t know what to do today, other than not having a phone, in order to get privacy and a phone that works without any BS.

bamboozled 2y ago
I immediately thought the same, I don’t want it. I personally don’t find ChatGPT very useful. Let’s just hope it’s opt out.
- hackernewds 2y ago
  Curious why not?
  - ghaff 2y ago
    Not the parent but I just don’t. I have used it to basically help with some largely boilerplate writing—as in get a pre-draft down on a page and adapt it from there. Don’t do a lot of programming any longer. It’s sometimes useful but at this point it just isn’t revolutionary for anything I do.
- tropicalbeach 2y ago
  People like this live in the stone age
  - wiseowise 2y ago
    >> I don’t want to use half-baked, heavily censored, hallucinating piece of crap that will affect battery life of my device > you’re living in the Stone Age, lolz
  - weikju 2y ago
    Your comment isn’t helpful to have a good and useful conversation about this. There are multiple viewpoints and levels of acceptance of new technology. Skepticism is healthy and even warranted. We don’t have to gleefully embrace every new shiny thing the tech companies throw at us. Heck the founder of this site himself once wrote an essay cautioning about adopting new services or tech too quickly because it is too often turned against us [1]: > It took a while though—on the order of 100 years. And unless the rate at which social antibodies evolve can increase to match the accelerating rate at which technological progress throws off new addictions, we'll be increasingly unable to rely on customs to protect us. [3] Unless we want to be canaries in the coal mine of each new addiction—the people whose sad example becomes a lesson to future generations—we'll have to figure out for ourselves what to avoid and how. It will actually become a reasonable strategy (or a more reasonable strategy) to suspect everything new. [1] https://paulgraham.com/addiction.html https://paulgraham.com/addiction.html

Apocryphon 2y ago

This is going to severely undercut Rabbit's "Large Action Model" shtick, isn't it

ENGNR 2y ago
Rabbits main play is really to bypass the App Store using the web interface for everything. Otherwise it would have just been an app.

userbinator 2y ago

It doesn't surprise me after seeing that Apple just did this: https://news.ycombinator.com/item?id=40313733 https://news.ycombinator.com/item?id=40313733 Also, is it just me or does the example of asking ChatGPT for gift ideas seem rather "dehumanising", for lack of a better term?

shepherdjerred 2y ago
How is this deal related to the iPad ad you linked?

devinprater 2y ago

I can't wait to see what accessibility updates they can do with local/remote AI! I'm definitely getting the Pro iPhone this year if it's running a faster CPU or has more RAM for bigger models. And that could be a selling point for the pro phones this year. Get AppleLLM 3B on a 6 GB iPhone, or Apple 7B which can describe images/videos right on device, describe animations in apps, and try to describe what's happening in all those emulated video games, with the 8/12GB iPhone Pro. I know, wild. But Apple's current screen recognition, using image classification ML from like 3 years ago, works a good 1 description per 2 seconds or so. If they can make AI updates that fast, with a buffer to only keep describing changes in the scene/video, I think it'd work well. On the other hand, isn't it a bit late for Apple to be finalizing things before WWDC? I mean, that doesn't give them that much time to make the video and prepare dev builds. And then they have to tune the system prompt, slide it into the OS (although maybe they have just an API field they can just plug it into), and then do QA. I don't know. Maybe this is old info. Still, I'll be eagerly awaiting any VoiceOver/accessibility updates!

osigurdson 2y ago

I used Siri for a while, then laughed how bad it was compared to Google, then turned it off (also unplugged Google home spying thing and Amazon spying thing, now in same box as Wii). Maybe this will turn things around, but I probably will never turn Siri on again as it will always been at best equally as good as ChatGPT, so why bother?

deleted 2y ago

[deleted]

r0fl 2y ago

Sam Altman on the all in podcast this week Such weird timing to do an interview and avoid on device versions for Apple If this Bloomberg rumour is true, why wouldn’t he schedule that interview for next week? What a waste of everyone’s time

ChrisArchitect 2y ago

Related: Apple Will Revamp Siri to Catch Up to Its Chatbot Competitors https://news.ycombinator.com/item?id=40322104 https://news.ycombinator.com/item?id=40322104