10 ms·
Apple's AI Strategy in a Nutshell
- Havoc 2y agoSurprised at the mention of TPUs. Apple used Google cloud for its training?
- sudhirj 2y agoThe the term TPU a Google trademark? I'm understanding it to mean any ML focussed ASIC or supplemental SOC hardware.
- Havoc 2y agoEveryone else calls them NPU. Except for a few edgy companies going for LPU TPU as best as I can see is always google. Either in cloud or their coral devices
- ZeroCool2u 2y agoI mean TPU's are generally the only off the shelf viable and reliable alternative to Nvidia GPU's for training and there are a lot of former Google staff at Apple that have experience with TPU's, so I wouldn't be surprised.
- msoad 2y agoThe article mentions that Apple can use OpenAI responses for further training their own models. I don’t want to say that’s impossible but that means they really screwed OpenAI in this deal. OpenAI terms don’t allow training on responses
- renegat0x0 2y agoWhile I agree I think Apple has a different sort of agreement. Big corporations can have different agreements between them.
- arvinsim 2y agoOn the contrary, it's worrying to wonder what Apple gave up to make OpenAI accept those terms.
- dymk 2y agoCash money, probably. Apple has a lot of it.
- visarga 2y agoThose 4o responses are also sensitive user data. Not ok for model dev
- elicksaur 2y ago>But Apple can also collect data on how users utilize GPT-4o versus Apple’s models, and perform gap assessment. This becomes valuable training data for Apple. I’d read that as: they know what queries users choose to route to OpenAI, so they can identify where their models are being perceived as less capable. I don’t think the author is revealing non-public knowledge of the contracts.
- msoad 2y agoThat makes sense!
- glimshe 2y agoApple's move is interesting, despite being a lot more reserved than this article's fanboy take on the impact of the move. For me, it is at least another clear blow on the "AI is just hype" arguments I still see around here.
- troupo 2y agoA yes, a "clear blow". Because we can definitely deduce it from... an over-produced video with sloppy AI-generated images and some heavily edited actions/responses from a product that will not be released until fall (and even then, not with all features). What other "clear blows" have there been so far that this one is "yet another"?
- elicksaur 2y agoIf you’re in the right info bubble, every AI announcement is a clear blow ;)
- ydnaclementine 2y agoAm I crazy to think that the claim of private cloud compute running M chips is marketing fluff? Is their ARM based SoC really as performant as a GPU at the scale they need?
- neilalexander 2y agoI think these days Apple Silicon is more of an umbrella term for the packaging of their CPUs, GPUs and NPUs.
- superlopuh 2y agoIf they wanted to, why wouldn't they make a modified, specialized, server rack version of their hardware?
- eddyg 2y agoInternally it’s Project ACDC – Apple Chips in Data Centers. https://www.wsj.com/tech/ai/apple-is-developing-ai-chips-for-data-centers-seeking-edge-in-arms-race-0bedd2b2 https://www.wsj.com/tech/ai/apple-is-developing-ai-chips-for...
- mkl 2y agohttps://archive.ph/p4iuu https://archive.ph/p4iuu No real details.
- steve1977 2y agoDoes it need to be? When they can use their own SoC servers, they don't have buy hardware from other vendors or rent cloud compute capacity. So it only needs to be so efficient that it will still cost less than those other options.
- xcv123 2y agoNo one said they are M chips. These would be custom made for servers and AI inferencing. Also note their "ARM based SoC" has a GPU with 800 GB/sec bandwidth in the M2 Ultra.
- stanislavb 2y ago"Indie app publishers are screwed." - that was one of the first things I thought about.
- detourdog 2y agoI didn’t understand this. If an indie developer provides an app a user pays for why should the UX choice of the user matter?
- sholladay 2y agoBecause these days a lot of the money a developer makes isn’t made up-front, it’s made through interactions in the app. Siri isn’t going to read you the ads in the apps it’s using behind the scenes. Maybe this will cause a shift back towards up-front pricing, which personally I would welcome.
- detourdog 2y agoThat is the only thing I could come up with up but I thought it was to cynical. The problem is that Indie is not the right description of those developers. Those developers come in all shapes and sizes.
- starbugs 2y agoApple's AI strategy in a nutshell: Sell more devices by using privacy and security as an argument for making new features available only on new iPhones.
- stanislavb 2y agoYup, this was the second thing I thought that's happening. Also, I wasn't sure whether to upgrade from 13 to 16. However, the AI angle tends everything towards buying a new phone. A very smart move from Apple.
- eddyg 2y agoIt’s misleading to not point out that Apple Intelligence is compatible with all devices with at least an M1 — which was announced in November 2020.
- deleted 2y ago[deleted]
- steve1977 2y agoOn the mobile phone side, it's compatible only with the latest iPhone 15 Pro (not even the standard iPhone 15).
- eddyg 2y agoThe parent comment makes it sound like everybody has to buy new hardware to use AI. Plenty of users with 2-3 year old hardware will be able to use it. As well as all the existing iPhone 15 Pro owners.
- mkl 2y ago> Apple is already a vertically integrated AI company, and deserve a higher valuation. Why does the latter follow from the former? I guess the author has Apple stock.
- m0llusk 2y agoThe larger context is that Apple gets most of its revenue from iPhone sales and use which have seen tepid growth. This strongly indicates reductions in valuation.
- wasteduniverse 2y ago[dead]
- LaserPineapple 2y ago"Also, Siri’s agentic features - if they work as advertised - can increase Apple’s leverage over App Publishers, because now the AI - not the user - is the entity opening and clicking on the apps." This was really interesting to me. How does one develop an app for Siri (or an AI agent in general). Is there a standard way to communicate and expose the functionality of your app?
- stavros 2y agoWe're going to have to pay to market our apps to an AI now, aren't we...
- IMTDb 2y ago> It’s now clear that Apple knows how to train frontier model-quality models, but it’s simply choosing to lay low. Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality I really don't see where this comes from. Apple has been deploying Siri for a decade. Despite this, Siri is still a steaming pile of cow dung. In few months, OpenAI built a working Siri, something that no-one at Apple was remotely close to achiving. The fact that Apple signed a deal with OpenAI and includes GPT-4o as an alternative option is a clear sign that Apple server-side models are really not anywhere near GPT-4o. If they were, Apple wouldn't have signed this deal which is so unlike them. To me, it really looks like Apple is late to the party in terms of LLM. They are betting that within a few years, high quality models will be commoditised and that having an ecosystem that leverages them properly will be the differentiator. Until then they are reluctantly incorporating the market leader in order not mis the train.
- troupo 2y ago> something that no-one at Apple was remotely close to achiving. Probably due to internal political wars. Apparently they had an internal lightweight version outperforming existing Siri, but the team never got anywhere with it: https://daringfireball.net/linked/2024/06/06/how-the-wall-street-journal-fell-behind-in-the-apple-is-behind-on-ai-arms-race https://daringfireball.net/linked/2024/06/06/how-the-wall-st...
- red2awn 2y agoThe article seems to be based on misinterpreted information: > Apple’s model has extremely low latency (0.6 milliseconds to first token), outperforms similar sized Phi and Gemini models from Microsoft and Google. From [1] it is 0.6 millisecond per prompt token so unless the prompt is one token the latency would be higher. > Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality The benchmarks from [1] never mentioned GPT-4o, the best model they compared to for the server model is GPT-4-0125. If their model almost matches GPT-4o they wouldn't need to integrate with ChatGPT. [1]: https://machinelearning.apple.com/research/introducing-apple-foundation-models https://machinelearning.apple.com/research/introducing-apple...
- deleted 2y ago[deleted]
- elicksaur 2y agoIf you stream the answer, the first token time is roughly the per token time.
- lostmsu 2y agoNo, you have to feed the entire prompt token-by-token before getting response.
- elicksaur 2y agoOh, I see, I misread. Thought it meant 0.6ms per output token. Now I get that it’s saying “prompt token”, so if your prompt is 100 tokens, that’s 60ms. That seems pretty fast. 1.6k tokens for a 1s time. Do other models compare to that? I’m not sure what the current top ranking for this metric looks like.
- lostmsu 2y agoLatency here is weird. You are using the number as bandwidth in this calculation. Perhaps reporter doesn't really know what he's talking about.
- JSR_FDED 2y agoApple has been laying the foundation for “agentic” use of apps for a long time. All of the functions that apps make available to Shortcuts today will be usable by Apple Intelligence. I wonder if they already had that use case in mind when they came out with Shortcuts?
- troupo 2y ago> I wonder if they already had that use case in mind when they came out with Shortcuts? They didn't come out with shortcuts. They bought an app called Workflow that was leagues ahead of anything Apple was providing on the automation front in iOS. And Apple never knew what to do with Shortcuts. They bought the app in 2017 and didn't integrate Siri with it until 2022. And Shortcuts still remain limited, and barely usable.