5 ms·

> The ChatGPT integration, powered by GPT-4o, will come to iOS, iPadOS, and macOS later this year. Jensen Huang must be having the time of his life right now.

> The ChatGPT integration, powered by GPT-4o, will come to iOS, iPadOS, and macOS later this year. Jensen Huang must be having the time of his life right now. Nvidia's relationship with Apple went from pariah to prodigal son, real fast.

rldjbpin 2y ago
given their history, he would only be satisfied when apple is forced to directly rely on nvidia hardware. current situation is like nvidia devs using macs at work giving mr cook some satisfaction or something.
dereg 2y ago
If we know anything about Apple, they're going after Nvidia. If anyone can pull it off, it's going to be them.
- ThinkBeat 2y ago
  I am of course wrong frequently, but I cannot see how that would happen. If they create cpu/gpus that are faster/better than what Nvidia sells, but they only sell them as part of a Mac desktop or laptop systems it wont really compete. For that they would have to develop servers that has a mass amount of whatever it is or sell the chips in the same manner Nvidia does today. I dont see that future for Apple. Microsoft / Google / or other major cloud companies would do extremely well if they could develop it and just keep it as a major win for their cloud products. Azure is running OpenAI as far as I have heard. Imagine if M$ made a crazy fast GPU/whatever. It would be a huge competitive advantage. Can it happen? I dont think so.
- talldayo 2y ago
  Well, good luck to Apple then. Hopefully this attempt at killing Nvidia goes better than the first time they tried, or when they tried and gave-up on making OpenCL. I just don't understand how they can compete on their own merits without purpose-built silicon; the M2 Ultra doesn't shine a candle to a single GB200. Once you consider how Nvidia's offerings are networked with Mellanox and CUDA universal memory, it feels like the only advantage Apple has in the space is setting their own prices. If they want to be competitive, I don't think they're going to be training Apple models on Apple Silicon.
  - 0xWTF 2y ago
    S&P 500 average P:E - 20 to 25 NASDAQ average P:E - 31 NVidia's P:E - 71 That's a market of 1 vendor. That's ripe for attack.
    - talldayo 2y ago
      Let's check in with OpenCL and see how far it got disrupting CUDA. You see, I want to live in a world where GPU manufacturers aren't perpetually hostile against each other. Even Nvidia would, judging by their decorum with Khronos. Unfortunately, some manufacturers would rather watch the world burn than work together for the common good. Even if a perfect CUDA replacement existed like it did with DXVK and DirectX, Apple will ignore and deny it while marketing something else to their customers. We've watched this happen for years, and it's why MacOS perennially cannot run many games or reliably support Open Source software. It is because Apple is an unreasonably fickle OEM, and their users constantly pay the price for Apple's arbitrary and unnecessary isolationism. Apple thinks they can disrupt AI? It's going to be like watching Stalin try to disrupt Wal-Mart.
      - labcomputer 2y ago
        > Let's check in with OpenCL and see how far it got disrupting CUDA. That's entirely the fault of AMD and Intel fumbling the ball in front of the other team's goal. For ages the only accelerated backend supported by PyTorch and TF was CUDA. Whose fault was that? Then there was buggy support for a subset of operations for a while. Then everyone stopped caring. Why I think it will go different this time: nVidia's competitors seem to have finally woken up and realized they need to support high level ML frameworks. "Apple Silicon" is essentially fully supported by PyTorch these days (via the "mps" backend). I've heard OpenCL works well now too, but have no hardware to test it on.
    - anvuong 2y ago
      It's ripe for attack. But Nvidia is still in its growing phase, not some incumbent behemoth. The way Nvidia ruthlessly handled AMD tell us that they are ready for competition.
- whimsicalism 2y ago
  i would strongly take the other side of that bet
  - dereg 2y ago
    Nvidia obviously has an enormous, enormous moat but I do think this is one of the areas in which Apple may actually GAF. The rollout of Apple Intelligence is going to make them the biggest provider of "edge" inference on day one. They're not going to be able to ride on optimism in services growth forever.
    - whimsicalism 2y ago
      Apple simply does not have the talent pool to take on either nvidia or the big LLM providers anywhere on the stack except for edge inference. If you're saying Apple is going to 'take on nvidia' in edge inference, then I don't disagree but I would hardly even count that as taking on nvidia.
      - dereg 2y ago
        I can't really dispute any of that. It took almost a decade but the PA Semi acquisition showed that Apple was able to get out of the shadow of its PowerPC era. Nvidia will remain a leader in this space for a long time. But things are going to play out wonky and Apple, when determined, are actually pretty good at executing on longer-term roadmaps.
  - Ancapistani 2y ago
    Personally, I'm taking _both_ sides of that bet. I think Apple is going to make rapid and substantial advancements in on-device AI-specific hardware. I also think nVIDIA is going to continue to dominate the cloud infrastructure space for training foundational models for the foreseeable future, and serving user-facing LLM workloads for a long time as well.
    - whimsicalism 2y ago
      edge inference? sure - but nvidia is not even a major player in that space now so i wouldn't really count that as 'taking on nvidia'.
- MR4D 2y ago
  Why do you think that? You seem to be positioning this as a Ford vs Chevy duel, when (to me at least) the comparison should be to Ford vs Exxon. Nvidia is an infrastructure company. And a darned good one. Apple is a user facing company and has outsourced infrastructure for decades (AWS & Azure being two of the well known ones).
  - dereg 2y ago
    Apple outsourced chips to IBM (PowerPC) for a long time and floundered all the while. They went into the game themselves w/ the PA Semi acquisition and now they have Apple Silicon to show for it.
    - MR4D 2y ago
      But Apple is vertically integrating. Thats like Ford buying Bridgestone. The only way it hurts Nvidia is if Apple becomes the runaway leader of the pc market. Even then, Apple hasn’t shown any intent of selling GPUs or AI processors to the likes of AWS, or Azure or Oracle, etc. Nvidia has a much bigger threat from with Intel/AMD or the cloud providers backward integrating and then not buying Nvidia chips. Again, no signs that Apple wants to do this.
- 01100011 2y ago
  Apple could have moved on Nvidia but instead they seem to have thrown in the towel and handed cash back to investors. The OpenAI deal seems like further admission by Apple that they missed the AI boat.
- gizmo 2y ago
  Exactly. Apple really needs new growth drivers and Nvidia has a 3bn market cap Apple wants to take a bite out of. One of the few huge tech growth areas that Apple can expand into.
cube2222 2y ago
Eh, it seems from the keynote that ChatGPT will be very selectively used, while most features will be powered by on-device processing and Apple's own private cloud running apple silicon. So all in all, not sure if it's that great for Nvidia.
- 01100011 2y ago
  If OpenAI is furiously buying GPUs to train larger models and Apple is handing OpenAI cash, then this seems like a win for Nvidia. You can argue about how big of a win, but it seems like a positive development. What would not have been positive for Nvidia is Apple saying they've adapted their HW to server chips and would be partnering with OpenAI to leverage them, but that didn't happen. Apple is busy handing cash back to investors and not seriously pursuing anything but inference.
ra7 2y ago
Didn’t Apple say they’re using their own hardware for serving some of the AI workloads? They dubbed it ‘Private Cloud Compute’. Not sure how much of a vote of confidence it is for Nvidia.
- whimsicalism 2y ago
  not for gpt4o workloads they aren't going to
  - jsheard 2y ago
    Plus even if Apple is using their own chips for inferencing, they're still driving more demand for training, which Nvidia still has locked down pretty tight.
    - ra7 2y ago
      Apple said they’re using their own silicon for training. Edit: unless I misunderstood and they meant only inference.
      - talldayo 2y ago
        They trained GPT-4o on Apple Silicon? I find that hard to believe, surely they only mean that some models were trained with Apple Silicon.
        ra7 2y ago
        Not GPT-4o, their own models that power some (most?) of the “Apple Intelligence” stuff.
      - jsheard 2y ago
        Interesting, I thought Apple Silicon mainly excelled at inferencing. Though I suppose the economics of it are unique for Apple themselves since they can fill racks full of barebones Apple Silicon boards without having to pay their own retail markup for complete assembled systems like everyone else does.
Whatarethese 2y ago
ChatGPT will only be invoked if on device and apple intelligence servers cant handle request.
- croes 2y ago
  To be useful Apple has to share the data with OpenAI
  - Workaccount2 2y ago
    I can only imagine Apple has some kind of siloing agreement with OpenAI, Apple can easily afford whatever price to do so.
    - talldayo 2y ago
      Surely Apple wouldn't simply market privacy while lying to their users about who can access their data: https://arstechnica.com/tech-policy/2023/12/apple-admits-to-secretly-giving-governments-push-notification-data/ https://arstechnica.com/tech-policy/2023/12/apple-admits-to-...
      - hnaccount_rng 2y ago
        There is a wide gap between complying with law enforcement requests and judicial orders and intentionally lying. Yes, if Apple can (trivially) read your data, then one must assume that at least the US government can access your data! Though if that's in your threat model I have a couple of other bad news items for you. Apple actively reduces that surface by moving ~everything to ee2e storage with keys held on customer devices. This is pretty transparently the attempt to be able to say "sorry can't do that without changing OS code and for _that_ discussion we have won in court. Really sorry that we can't help you". And yes, that's probably just to decrease the compliance costs. Still same result
    - noahtallen 2y ago
      Yes, also covered explicitly in the keynote that Apple user's requests to openAI are not tracked. (Plus you have the explicit opt-in to even access chatGPT via siri in the first place.)
KeplerBoy 2y ago
Not sure Nvidia is too happy with Apple. They are the first ones to ship on-device inference at scale on non-nvidia hardware. Apple also has the means to build data center training hardware using apple silicon if they want to do so. If they are serious about the OAI partnership they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests.
- whimsicalism 2y ago
  > Apple also has the means to build data center training hardware using apple silicon if they want to do so. i'm seeing people all over this thread saying stuff like that, it reads like fantasyland to me. Apple doesn't have the talent or the chips or suppliers or really any of the capabilities to do this, where are people getting it from?
  - KeplerBoy 2y ago
    Apple is already one of the largest (if not the largest) customers of TSMC and they have plenty of experience designing some of the best chips on the most modern nodes. Their ability to design a chip and networking fabric which is fast/efficient at training a narrow set of model architecture is not far fetched by any means.
    - talldayo 2y ago
      It's worth noting that one of Apple's largest competitor at TSMC is, in fact, Nvidia. And when you line the benchmarks up, Nvidia is one of the few companies that consistently beats Apple on performance-per-watt even when they aren't on the same TSMC node: https://browser.geekbench.com/opencl-benchmarks https://browser.geekbench.com/opencl-benchmarks
- mistersquid 2y ago
  > Apple also has the means to build data center training hardware using apple silicon if they want to do so. > If they are serious about the OAI partnership they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests Apple addressed both these points in today’s preso. 1. They will send requests that require larger contexts to their own Apple Silicon-based servers that will provide Apple devices a new product platform called Private Cloud Compute. 2. Apple’s OS generative AI request APIs won’t even talk to cloud compute resources that do not attest to infrastructure that has a publicly available privacy audit.
deleted 2y ago
[deleted]
swatcoder 2y ago
Apple's put ChatGPT integration on the very edge of Apple Intelligence. It's a win for OpenAI to have secured that opportunity, and Nvidia wins by extension (as long as OpenAI continues to rely on them themselves), but the vast majority of what Apple announced today appears to run entirely on Apple Silicon. It's not especially big news for Nvidia at all.

rldjbpin 2y ago

given their history, he would only be satisfied when apple is forced to directly rely on nvidia hardware. current situation is like nvidia devs using macs at work giving mr cook some satisfaction or something.

dereg 2y ago

If we know anything about Apple, they're going after Nvidia. If anyone can pull it off, it's going to be them.

ThinkBeat 2y ago
I am of course wrong frequently, but I cannot see how that would happen. If they create cpu/gpus that are faster/better than what Nvidia sells, but they only sell them as part of a Mac desktop or laptop systems it wont really compete. For that they would have to develop servers that has a mass amount of whatever it is or sell the chips in the same manner Nvidia does today. I dont see that future for Apple. Microsoft / Google / or other major cloud companies would do extremely well if they could develop it and just keep it as a major win for their cloud products. Azure is running OpenAI as far as I have heard. Imagine if M$ made a crazy fast GPU/whatever. It would be a huge competitive advantage. Can it happen? I dont think so.
talldayo 2y ago
Well, good luck to Apple then. Hopefully this attempt at killing Nvidia goes better than the first time they tried, or when they tried and gave-up on making OpenCL. I just don't understand how they can compete on their own merits without purpose-built silicon; the M2 Ultra doesn't shine a candle to a single GB200. Once you consider how Nvidia's offerings are networked with Mellanox and CUDA universal memory, it feels like the only advantage Apple has in the space is setting their own prices. If they want to be competitive, I don't think they're going to be training Apple models on Apple Silicon.
- 0xWTF 2y ago
  S&P 500 average P:E - 20 to 25 NASDAQ average P:E - 31 NVidia's P:E - 71 That's a market of 1 vendor. That's ripe for attack.
  - talldayo 2y ago
    Let's check in with OpenCL and see how far it got disrupting CUDA. You see, I want to live in a world where GPU manufacturers aren't perpetually hostile against each other. Even Nvidia would, judging by their decorum with Khronos. Unfortunately, some manufacturers would rather watch the world burn than work together for the common good. Even if a perfect CUDA replacement existed like it did with DXVK and DirectX, Apple will ignore and deny it while marketing something else to their customers. We've watched this happen for years, and it's why MacOS perennially cannot run many games or reliably support Open Source software. It is because Apple is an unreasonably fickle OEM, and their users constantly pay the price for Apple's arbitrary and unnecessary isolationism. Apple thinks they can disrupt AI? It's going to be like watching Stalin try to disrupt Wal-Mart.
    - labcomputer 2y ago
      > Let's check in with OpenCL and see how far it got disrupting CUDA. That's entirely the fault of AMD and Intel fumbling the ball in front of the other team's goal. For ages the only accelerated backend supported by PyTorch and TF was CUDA. Whose fault was that? Then there was buggy support for a subset of operations for a while. Then everyone stopped caring. Why I think it will go different this time: nVidia's competitors seem to have finally woken up and realized they need to support high level ML frameworks. "Apple Silicon" is essentially fully supported by PyTorch these days (via the "mps" backend). I've heard OpenCL works well now too, but have no hardware to test it on.
  - anvuong 2y ago
    It's ripe for attack. But Nvidia is still in its growing phase, not some incumbent behemoth. The way Nvidia ruthlessly handled AMD tell us that they are ready for competition.
whimsicalism 2y ago
i would strongly take the other side of that bet
- dereg 2y ago
  Nvidia obviously has an enormous, enormous moat but I do think this is one of the areas in which Apple may actually GAF. The rollout of Apple Intelligence is going to make them the biggest provider of "edge" inference on day one. They're not going to be able to ride on optimism in services growth forever.
  - whimsicalism 2y ago
    Apple simply does not have the talent pool to take on either nvidia or the big LLM providers anywhere on the stack except for edge inference. If you're saying Apple is going to 'take on nvidia' in edge inference, then I don't disagree but I would hardly even count that as taking on nvidia.
    - dereg 2y ago
      I can't really dispute any of that. It took almost a decade but the PA Semi acquisition showed that Apple was able to get out of the shadow of its PowerPC era. Nvidia will remain a leader in this space for a long time. But things are going to play out wonky and Apple, when determined, are actually pretty good at executing on longer-term roadmaps.
- Ancapistani 2y ago
  Personally, I'm taking _both_ sides of that bet. I think Apple is going to make rapid and substantial advancements in on-device AI-specific hardware. I also think nVIDIA is going to continue to dominate the cloud infrastructure space for training foundational models for the foreseeable future, and serving user-facing LLM workloads for a long time as well.
  - whimsicalism 2y ago
    edge inference? sure - but nvidia is not even a major player in that space now so i wouldn't really count that as 'taking on nvidia'.
MR4D 2y ago
Why do you think that? You seem to be positioning this as a Ford vs Chevy duel, when (to me at least) the comparison should be to Ford vs Exxon. Nvidia is an infrastructure company. And a darned good one. Apple is a user facing company and has outsourced infrastructure for decades (AWS & Azure being two of the well known ones).
- dereg 2y ago
  Apple outsourced chips to IBM (PowerPC) for a long time and floundered all the while. They went into the game themselves w/ the PA Semi acquisition and now they have Apple Silicon to show for it.
  - MR4D 2y ago
    But Apple is vertically integrating. Thats like Ford buying Bridgestone. The only way it hurts Nvidia is if Apple becomes the runaway leader of the pc market. Even then, Apple hasn’t shown any intent of selling GPUs or AI processors to the likes of AWS, or Azure or Oracle, etc. Nvidia has a much bigger threat from with Intel/AMD or the cloud providers backward integrating and then not buying Nvidia chips. Again, no signs that Apple wants to do this.
01100011 2y ago
Apple could have moved on Nvidia but instead they seem to have thrown in the towel and handed cash back to investors. The OpenAI deal seems like further admission by Apple that they missed the AI boat.
gizmo 2y ago
Exactly. Apple really needs new growth drivers and Nvidia has a 3bn market cap Apple wants to take a bite out of. One of the few huge tech growth areas that Apple can expand into.

cube2222 2y ago

Eh, it seems from the keynote that ChatGPT will be very selectively used, while most features will be powered by on-device processing and Apple's own private cloud running apple silicon. So all in all, not sure if it's that great for Nvidia.

01100011 2y ago
If OpenAI is furiously buying GPUs to train larger models and Apple is handing OpenAI cash, then this seems like a win for Nvidia. You can argue about how big of a win, but it seems like a positive development. What would not have been positive for Nvidia is Apple saying they've adapted their HW to server chips and would be partnering with OpenAI to leverage them, but that didn't happen. Apple is busy handing cash back to investors and not seriously pursuing anything but inference.

ra7 2y ago

Didn’t Apple say they’re using their own hardware for serving some of the AI workloads? They dubbed it ‘Private Cloud Compute’. Not sure how much of a vote of confidence it is for Nvidia.

whimsicalism 2y ago
not for gpt4o workloads they aren't going to
- jsheard 2y ago
  Plus even if Apple is using their own chips for inferencing, they're still driving more demand for training, which Nvidia still has locked down pretty tight.
  - ra7 2y ago
    Apple said they’re using their own silicon for training. Edit: unless I misunderstood and they meant only inference.
    - talldayo 2y ago
      They trained GPT-4o on Apple Silicon? I find that hard to believe, surely they only mean that some models were trained with Apple Silicon.
      - ra7 2y ago
        Not GPT-4o, their own models that power some (most?) of the “Apple Intelligence” stuff.
    - jsheard 2y ago
      Interesting, I thought Apple Silicon mainly excelled at inferencing. Though I suppose the economics of it are unique for Apple themselves since they can fill racks full of barebones Apple Silicon boards without having to pay their own retail markup for complete assembled systems like everyone else does.

Whatarethese 2y ago

ChatGPT will only be invoked if on device and apple intelligence servers cant handle request.

croes 2y ago
To be useful Apple has to share the data with OpenAI
- Workaccount2 2y ago
  I can only imagine Apple has some kind of siloing agreement with OpenAI, Apple can easily afford whatever price to do so.
  - talldayo 2y ago
    Surely Apple wouldn't simply market privacy while lying to their users about who can access their data: https://arstechnica.com/tech-policy/2023/12/apple-admits-to-secretly-giving-governments-push-notification-data/ https://arstechnica.com/tech-policy/2023/12/apple-admits-to-...
    - hnaccount_rng 2y ago
      There is a wide gap between complying with law enforcement requests and judicial orders and intentionally lying. Yes, if Apple can (trivially) read your data, then one must assume that at least the US government can access your data! Though if that's in your threat model I have a couple of other bad news items for you. Apple actively reduces that surface by moving ~everything to ee2e storage with keys held on customer devices. This is pretty transparently the attempt to be able to say "sorry can't do that without changing OS code and for _that_ discussion we have won in court. Really sorry that we can't help you". And yes, that's probably just to decrease the compliance costs. Still same result
  - noahtallen 2y ago
    Yes, also covered explicitly in the keynote that Apple user's requests to openAI are not tracked. (Plus you have the explicit opt-in to even access chatGPT via siri in the first place.)

KeplerBoy 2y ago

Not sure Nvidia is too happy with Apple. They are the first ones to ship on-device inference at scale on non-nvidia hardware. Apple also has the means to build data center training hardware using apple silicon if they want to do so. If they are serious about the OAI partnership they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests.

whimsicalism 2y ago
> Apple also has the means to build data center training hardware using apple silicon if they want to do so. i'm seeing people all over this thread saying stuff like that, it reads like fantasyland to me. Apple doesn't have the talent or the chips or suppliers or really any of the capabilities to do this, where are people getting it from?
- KeplerBoy 2y ago
  Apple is already one of the largest (if not the largest) customers of TSMC and they have plenty of experience designing some of the best chips on the most modern nodes. Their ability to design a chip and networking fabric which is fast/efficient at training a narrow set of model architecture is not far fetched by any means.
  - talldayo 2y ago
    It's worth noting that one of Apple's largest competitor at TSMC is, in fact, Nvidia. And when you line the benchmarks up, Nvidia is one of the few companies that consistently beats Apple on performance-per-watt even when they aren't on the same TSMC node: https://browser.geekbench.com/opencl-benchmarks https://browser.geekbench.com/opencl-benchmarks
mistersquid 2y ago
> Apple also has the means to build data center training hardware using apple silicon if they want to do so. > If they are serious about the OAI partnership they could also start to supply them with cloud inference hardware and strongarm them into only using apple servers to serve iOS requests Apple addressed both these points in today’s preso. 1. They will send requests that require larger contexts to their own Apple Silicon-based servers that will provide Apple devices a new product platform called Private Cloud Compute. 2. Apple’s OS generative AI request APIs won’t even talk to cloud compute resources that do not attest to infrastructure that has a publicly available privacy audit.

deleted 2y ago

[deleted]

swatcoder 2y ago

Apple's put ChatGPT integration on the very edge of Apple Intelligence. It's a win for OpenAI to have secured that opportunity, and Nvidia wins by extension (as long as OpenAI continues to rely on them themselves), but the vast majority of what Apple announced today appears to run entirely on Apple Silicon. It's not especially big news for Nvidia at all.