7 ms·

Apple really dropped the ball here. They had every ability to make something competitive with Nvidia for AI training as well as inference, by selling high end m

Apple really dropped the ball here. They had every ability to make something competitive with Nvidia for AI training as well as inference, by selling high end multi GPU Mac Pro workstations as well as servers, but for some reason chose not to. They had the infrastructure and custom SoCs and everything. What a waste. It really could have been a bigger market for them than even the iPhone.

etchalon 7mo ago
Nothing is a bigger market than the iPhone, let alone expensive niche machines.
A_D_E_P_T 7mo ago
Just about everybody who isn't Nvidia dropped the ball, bigtime. Intel should have shipped their GPUs with much more VRAM from day one. If they had done this, they'd have carved out a massive niche and much more market share, and it would have been trivially simple to do. AMD should have improved their tools and software, etc. Apple should have done as you say. Google had nigh on a decade to boost TPU production, and they're still somehow behind the curve. Such a lack of vision. And thus Nvidia is, now quite durably, the most valuable company in the world. Imagine telling that to a time traveler from 2018.
- readitalready 7mo ago
  I think for AMD, they were focused on competing against Intel. Remember AMD was almost bankrupt about 15 years ago because of competing against Intel. But the very first GPU use for AI was actually with an ATI/AMD GPU, not an Nvidia one. Everyone thinks Nvidia kicked off the GPU AI craze when Ilya Sutskever cleaned up on AlexNet with an Nvidia GPU back in 2012, or when Andrew Ng and team at Stanford published their "Large Scale Deep Unsupervised Learning using Graphics Processors" in 2009, but in 2004, a couple of Korean researchers were the first to implement neural networks on a GPU, using ATI Radeons: https://www.sciencedirect.com/science/article/abs/pii/S0031320304000524 https://www.sciencedirect.com/science/article/abs/pii/S00313... And as of now I do believe AMD is in the second strongest position in the datacenter space after Nvidia, ahead of even Google.
- bigstrat2003 7mo ago
  > And thus Nvidia is, now quite durably, the most valuable company in the world. Nvidia is the most valuable company in the world right up until the AI bubble pops. Which, while it's hard to nail down when, is going to happen. I wouldn't call their position durable at all.
  - HerbManic 7mo ago
    It is definitely a case that they will fall a long way but Nvidia will not fail as a whole. They have a way of maximizing their position relentlessly. CUDA turns out to endlessly put them in amazing positions on things like image recognition, AR, Crypto and now AI. For all the faults of them leaning in hard on these things for stock market and personal gains, Nvidia still has some of the best quality products around. That is their saving grace. They will not be the world most valuable company once the bubble pops, will probably never get back there again, but they will continue to be a decent enough business. I just want them going back to talking about graphics more than AI again, that will be nice.
  - gizajob 7mo ago
    The crashing and burning of Nvidia stock has been predicted for a while now and keeps not really happening. It’s gone pretty flat and volatile up there around $180 but they keep delivering the results to back it up. I was thinking this week that Apple is really primed to make a killing from people who want to run their LLM on-device coupled with an agent in the next couple of years. We’re a long way off being able to train the models – this is going to need an Nvidia-powered datacentre for the foreseeable future, but the local inference seems absolutely like a market that Apple could capture, gutting all the most premium revenue from Anthropic and OpenAI by selling Macs with a large amount of integrated memory to anyone who wants to give them the money to run their native OpenClaw/agent instead of paying ever-growing monthly bills for tokens.
  - user34283 7mo ago
    I might as well say that no, it is not going to happen. As handwriting code is rapidly going out of fashion this year, it seems likely AI is coming for most of knowledge work next. And who is to say that manual labor is safe for long?
- gehsty 7mo ago
  Why should Apple have done this? It doesn’t fit their business in anyway shape or form. Where does data centre hardware sit relative to electronics / humanities cross roads that is foundational for Apple?
  - AdamN 7mo ago
    Yeah nothing about Apple is server side and imho that's what training is. To be serious about it as a company you have all sorts of other tools (crawlers, etc...) helping with training so it basically has to be in the datacenter at any reasonable scale anyway. And that's just not where Apple lives. We saw with Swift that they couldn't focus on server side enough to make it a serious language there and they've consistently declined to enter that area over the years because it's outside their wheelhouse.
  - DCKing 7mo ago
    > Why should Apple have done this? For money, probably. Apple is presumably leaving a lot of money on the table by not trying to sell Apple Silicon for AI inference and training. They're the only ones who can attach reasonably large GPUs (M3 Ultra) to very large amounts of cheaper memory (512GB SO-DIMM per GPU). Apple could e.g. sell server SKUs of Mac Studios, heck they can sell M3 Ultra chips on PCIe cards. And they could further develop Apple Silicon in that direction. Presumably they would be seen as a very legit competitor to Nvidia that way, perhaps moreso than Intel and AMD. I'd assume that in the current climate this would be extremely lucrative. Now, actually doing this would disrupt Apple's own supply chain as well as force it to spend significant internal resources and cultural change for this kind of product line. There's a good argument to be made it would disproportionally negatively affect its Mac business, so this would be a very risky move. But given that AI hardware is likely much higher margin than the Mac business an argument could probably (sadly) be made that it'd be lucrative for them to try it. I personally don't think Apple is inclined to take this kind of risk to jeopardize the Mac, but I'm sure some people at Apple have considered this.
    - gehsty 7mo ago
      I guess I mean for apple to remain as apple, they would not do this due to company culture.
- 1W6MIC49CYX9GAP 7mo ago
  Intel doesn't limit how much memory card makers can pair with their GPU. It's up to the card maker.
- BeetleB 7mo ago
  Trust me: If Intel could, it would. From inside news: They were not breaking even on their existing GPUs. The strategy was to take a loss just to have a presence in the space.
  - dabockster 7mo ago
    Intel could position their cards as strong for certain workloads. They had AV1 support first in market, for example.
zer00eyz 7mo ago
> something competitive with Nvidia for AI training Apple is counting on something else: model shrink. Every one is now looking at "how do we make these smaller". At some point a beefy Mac Studio and the "right sized" model is going to be what people want. Apple dumped a 4 pack of them in the hands of a lot of tech influencers a few months back and they were fairly interesting (expensive tho).
- Forgeties79 7mo ago
  Cheaper than what you’d expect though. You could get a nice setup for $20-40k 6mo ago. As far as enterprise investments go, that’s a rounding error.
  - zer00eyz 7mo ago
    Drop that down to 5k, and make it useful. Give every iPhone family a in house Siri that will deal with canceling services and pursuing refunds. Your customer screw up results in your site getting an agent drive DDOS on its CS department till you give in. Siri: "Hey User, here's your daily update, I see you haven't been to the gym, would you like me to harass their customer service department till they let you out of their onerous contract?"
    - Forgeties79 7mo ago
      I’m running modest setup using a mistral model (24B) on a 9070 (AMD) and 32gb of ram. $1800 machine at the time I built it. It ultimately boils down to what you want to do with it. For me, it’s basically a drafting tool. I use it to break through writer’s block, iterate, or just throw out some ideas. Sometimes summarize but that can be hit or misss. I don’t need the latest and greatest and I fine tuned LM studio enough that I get acceptable results in 30 to 90 seconds that help me keep moving ahead. I am not a software engineer, I am definitely not as much of a “coder” as the average person on HN. So if I can do it for less than $2000, I bet a lot of (smarter/experience coding) people could see great results for $5000. You can get an M3 ultra Mac studio with 96gb ram for $4000. If you’re willing to go up to $6k it’s 256gb. Wayyyyy more firepower than my setup. I imagine plenty powerful for a lot of people.
vlovich123 7mo ago
Don’t mistake stock market performance for revenue. NVIDIA makes ~200B annually, same as what Apple makes from iPhones. It’s a big market but GPUs aren’t just AI.
- readitalready 7mo ago
  I'm purely talking in terms of revenue. There's a huge demand for AI systems from personal workstations to datacenter servers, and Apple was one of the few companies in the world in a position to build complete systems for it. But for some reason Apple thought the sound recording engineer or the video editor market was more important... like, WTF dude? Have some vision at least!
  - aurareturn 7mo ago
    Some people at Apple see it. That’s why they added matmul to M5 GPU and keep mentioning LMStudio in their marketing.
    - rudedogg 7mo ago
      Their rule of only releasing major software updates once a year in June is holding them back IMO. Their local LLM apis were dated before macOS/iOS 26 was even released. Just because something worked 20 years ago doesn’t mean it works today, but I’m sure it’s hard to argue against a historically successful strategy internally.
      - aurareturn 7mo ago
        Huh? What local LLM apis? It uses Metal.
        rudedogg 7mo ago
        The application development APIs, ie: https://developer.apple.com/documentation/technologyoverviews/foundation-models https://developer.apple.com/documentation/technologyoverview...
root_axis 7mo ago
Nah, Apple made the right choice. Nobody except a niche market of hobbyists is interested in running tiny quantized models.
- gizajob 7mo ago
  About the same niche market as the people who bought the Apple I, and we know where that went.
  - wtallis 7mo ago
    The Apple I was a pretty poor predictor of what mainstream mass-market computing was going to end up looking like. I don't think anybody has yet come up with the Apple II of local LLMs, let alone the VisiCalc or Windows 95.
- aurareturn 7mo ago
  Small models keep getting smarter and local hardware keeps getting better. At some point, they will converge and an inflection for local LLMs will happen. Local LLMs will never be as smart or fast as cloud LLMs but they will be very useful for lower value tasks.
bschwindHN 7mo ago
> They had the infrastructure and custom SoCs and everything. What a waste. What are they wasting, exactly?
kristopolous 7mo ago
this is what needs to come back with modern hardware and modern interconnect https://en.wikipedia.org/wiki/Xserve https://en.wikipedia.org/wiki/Xserve
zer0zzz 7mo ago
How is this dropping the ball? I think they dropped the ball a long time ago by waiting until M5 to do integrated tensor cores instead of the separate ANE only which was present before. For multi-gpu you can network multiple Macs at high speed now. Their biggest disadvantage to Nvidia right now is that no one wants to do kernel authoring in Metal. AMD learned that the hard way when they gave up on OpenCL and built HIP.
Almondsetat 7mo ago
If Apple doesn't offer a Linux product, they cannot be used seriously in headless computing task. They are adamant in controlling the whole stack, so unless they remake some server version of macOS (and wait years for the community to accustom themselves with it), they will keep being a consumer/professional oriented company
greggsy 7mo ago
They didn’t drop the ball at all? They want to be able to sell handsets, desktops and laptops to their customer base. Pursing a product line that would consume the finite amount of silicon manufacturing resources away from that user base would be corporate suicide. Even nvidia has all but dropped support for its traditional gaming customer base to satisfy its new strategy. At any rate, the local inference capabilities are only going to get cheaper and more accessible over the coming years, and Apple are probably better placed than anyone to make it happen.
gehsty 7mo ago
If my Grandma had wheels she would be a bicycle. Apple would need to transition from being a consumer electronics company to being a B2B retailer for data centre hardware to take advantage of this. Obviously Siri from WWDC 2yrs ago was a disaster for Apple. Other than that they seem to have done pretty well navigating the new LLM world. I do think they would benefit from having their own SOA LLM, but I don’t think its is necessary for them. My mental model for LLMs and Apple is that they are similar Garage Band - “Now everyone can play an instrument” becomes “now anyone can make an app”. Apple owns the interface to the user (i don’t see anyone making nicer to use consumer hardware) and can use what ever stack in the background to deliver the technical features they decide to.
EagnaIonat 7mo ago
> AI training as well as inference Inference has never been an issue for M series, and MLX just ramped it up further. You can do training on the latest MBPs, although any serious models you are going to the cloud anyway.
huslage 7mo ago
Apple makes AI inference and training servers by the thousands. They just don't sell them to anyone. They use them internally in their datacenters. They didn't drop the ball, they are playing a different game while not cannibalizing their existing customer base.
NetMageSCW 7mo ago
Apple didn’t drop the ball - they have no interest in creating servers for a limited time bubble. It is laughable that anyone would think that market would be bigger than iPhone - there’s a reason no RAM manufacturer is building new plants to take advantage of the current demand - they don’t expect it to last long enough to pay for their investment.

etchalon 7mo ago

Nothing is a bigger market than the iPhone, let alone expensive niche machines.

A_D_E_P_T 7mo ago

Just about everybody who isn't Nvidia dropped the ball, bigtime. Intel should have shipped their GPUs with much more VRAM from day one. If they had done this, they'd have carved out a massive niche and much more market share, and it would have been trivially simple to do. AMD should have improved their tools and software, etc. Apple should have done as you say. Google had nigh on a decade to boost TPU production, and they're still somehow behind the curve. Such a lack of vision. And thus Nvidia is, now quite durably, the most valuable company in the world. Imagine telling that to a time traveler from 2018.

readitalready 7mo ago
I think for AMD, they were focused on competing against Intel. Remember AMD was almost bankrupt about 15 years ago because of competing against Intel. But the very first GPU use for AI was actually with an ATI/AMD GPU, not an Nvidia one. Everyone thinks Nvidia kicked off the GPU AI craze when Ilya Sutskever cleaned up on AlexNet with an Nvidia GPU back in 2012, or when Andrew Ng and team at Stanford published their "Large Scale Deep Unsupervised Learning using Graphics Processors" in 2009, but in 2004, a couple of Korean researchers were the first to implement neural networks on a GPU, using ATI Radeons: https://www.sciencedirect.com/science/article/abs/pii/S0031320304000524 https://www.sciencedirect.com/science/article/abs/pii/S00313... And as of now I do believe AMD is in the second strongest position in the datacenter space after Nvidia, ahead of even Google.
bigstrat2003 7mo ago
> And thus Nvidia is, now quite durably, the most valuable company in the world. Nvidia is the most valuable company in the world right up until the AI bubble pops. Which, while it's hard to nail down when, is going to happen. I wouldn't call their position durable at all.
- HerbManic 7mo ago
  It is definitely a case that they will fall a long way but Nvidia will not fail as a whole. They have a way of maximizing their position relentlessly. CUDA turns out to endlessly put them in amazing positions on things like image recognition, AR, Crypto and now AI. For all the faults of them leaning in hard on these things for stock market and personal gains, Nvidia still has some of the best quality products around. That is their saving grace. They will not be the world most valuable company once the bubble pops, will probably never get back there again, but they will continue to be a decent enough business. I just want them going back to talking about graphics more than AI again, that will be nice.
- gizajob 7mo ago
  The crashing and burning of Nvidia stock has been predicted for a while now and keeps not really happening. It’s gone pretty flat and volatile up there around $180 but they keep delivering the results to back it up. I was thinking this week that Apple is really primed to make a killing from people who want to run their LLM on-device coupled with an agent in the next couple of years. We’re a long way off being able to train the models – this is going to need an Nvidia-powered datacentre for the foreseeable future, but the local inference seems absolutely like a market that Apple could capture, gutting all the most premium revenue from Anthropic and OpenAI by selling Macs with a large amount of integrated memory to anyone who wants to give them the money to run their native OpenClaw/agent instead of paying ever-growing monthly bills for tokens.
- user34283 7mo ago
  I might as well say that no, it is not going to happen. As handwriting code is rapidly going out of fashion this year, it seems likely AI is coming for most of knowledge work next. And who is to say that manual labor is safe for long?
gehsty 7mo ago
Why should Apple have done this? It doesn’t fit their business in anyway shape or form. Where does data centre hardware sit relative to electronics / humanities cross roads that is foundational for Apple?
- AdamN 7mo ago
  Yeah nothing about Apple is server side and imho that's what training is. To be serious about it as a company you have all sorts of other tools (crawlers, etc...) helping with training so it basically has to be in the datacenter at any reasonable scale anyway. And that's just not where Apple lives. We saw with Swift that they couldn't focus on server side enough to make it a serious language there and they've consistently declined to enter that area over the years because it's outside their wheelhouse.
- DCKing 7mo ago
  > Why should Apple have done this? For money, probably. Apple is presumably leaving a lot of money on the table by not trying to sell Apple Silicon for AI inference and training. They're the only ones who can attach reasonably large GPUs (M3 Ultra) to very large amounts of cheaper memory (512GB SO-DIMM per GPU). Apple could e.g. sell server SKUs of Mac Studios, heck they can sell M3 Ultra chips on PCIe cards. And they could further develop Apple Silicon in that direction. Presumably they would be seen as a very legit competitor to Nvidia that way, perhaps moreso than Intel and AMD. I'd assume that in the current climate this would be extremely lucrative. Now, actually doing this would disrupt Apple's own supply chain as well as force it to spend significant internal resources and cultural change for this kind of product line. There's a good argument to be made it would disproportionally negatively affect its Mac business, so this would be a very risky move. But given that AI hardware is likely much higher margin than the Mac business an argument could probably (sadly) be made that it'd be lucrative for them to try it. I personally don't think Apple is inclined to take this kind of risk to jeopardize the Mac, but I'm sure some people at Apple have considered this.
  - gehsty 7mo ago
    I guess I mean for apple to remain as apple, they would not do this due to company culture.
1W6MIC49CYX9GAP 7mo ago
Intel doesn't limit how much memory card makers can pair with their GPU. It's up to the card maker.
BeetleB 7mo ago
Trust me: If Intel could, it would. From inside news: They were not breaking even on their existing GPUs. The strategy was to take a loss just to have a presence in the space.
- dabockster 7mo ago
  Intel could position their cards as strong for certain workloads. They had AV1 support first in market, for example.

zer00eyz 7mo ago

> something competitive with Nvidia for AI training Apple is counting on something else: model shrink. Every one is now looking at "how do we make these smaller". At some point a beefy Mac Studio and the "right sized" model is going to be what people want. Apple dumped a 4 pack of them in the hands of a lot of tech influencers a few months back and they were fairly interesting (expensive tho).

Forgeties79 7mo ago
Cheaper than what you’d expect though. You could get a nice setup for $20-40k 6mo ago. As far as enterprise investments go, that’s a rounding error.
- zer00eyz 7mo ago
  Drop that down to 5k, and make it useful. Give every iPhone family a in house Siri that will deal with canceling services and pursuing refunds. Your customer screw up results in your site getting an agent drive DDOS on its CS department till you give in. Siri: "Hey User, here's your daily update, I see you haven't been to the gym, would you like me to harass their customer service department till they let you out of their onerous contract?"
  - Forgeties79 7mo ago
    I’m running modest setup using a mistral model (24B) on a 9070 (AMD) and 32gb of ram. $1800 machine at the time I built it. It ultimately boils down to what you want to do with it. For me, it’s basically a drafting tool. I use it to break through writer’s block, iterate, or just throw out some ideas. Sometimes summarize but that can be hit or misss. I don’t need the latest and greatest and I fine tuned LM studio enough that I get acceptable results in 30 to 90 seconds that help me keep moving ahead. I am not a software engineer, I am definitely not as much of a “coder” as the average person on HN. So if I can do it for less than $2000, I bet a lot of (smarter/experience coding) people could see great results for $5000. You can get an M3 ultra Mac studio with 96gb ram for $4000. If you’re willing to go up to $6k it’s 256gb. Wayyyyy more firepower than my setup. I imagine plenty powerful for a lot of people.

vlovich123 7mo ago

Don’t mistake stock market performance for revenue. NVIDIA makes ~200B annually, same as what Apple makes from iPhones. It’s a big market but GPUs aren’t just AI.

readitalready 7mo ago
I'm purely talking in terms of revenue. There's a huge demand for AI systems from personal workstations to datacenter servers, and Apple was one of the few companies in the world in a position to build complete systems for it. But for some reason Apple thought the sound recording engineer or the video editor market was more important... like, WTF dude? Have some vision at least!
- aurareturn 7mo ago
  Some people at Apple see it. That’s why they added matmul to M5 GPU and keep mentioning LMStudio in their marketing.
  - rudedogg 7mo ago
    Their rule of only releasing major software updates once a year in June is holding them back IMO. Their local LLM apis were dated before macOS/iOS 26 was even released. Just because something worked 20 years ago doesn’t mean it works today, but I’m sure it’s hard to argue against a historically successful strategy internally.
    - aurareturn 7mo ago
      Huh? What local LLM apis? It uses Metal.
      - rudedogg 7mo ago
        The application development APIs, ie: https://developer.apple.com/documentation/technologyoverviews/foundation-models https://developer.apple.com/documentation/technologyoverview...

root_axis 7mo ago

Nah, Apple made the right choice. Nobody except a niche market of hobbyists is interested in running tiny quantized models.

gizajob 7mo ago
About the same niche market as the people who bought the Apple I, and we know where that went.
- wtallis 7mo ago
  The Apple I was a pretty poor predictor of what mainstream mass-market computing was going to end up looking like. I don't think anybody has yet come up with the Apple II of local LLMs, let alone the VisiCalc or Windows 95.
aurareturn 7mo ago
Small models keep getting smarter and local hardware keeps getting better. At some point, they will converge and an inflection for local LLMs will happen. Local LLMs will never be as smart or fast as cloud LLMs but they will be very useful for lower value tasks.

bschwindHN 7mo ago

> They had the infrastructure and custom SoCs and everything. What a waste. What are they wasting, exactly?

kristopolous 7mo ago

this is what needs to come back with modern hardware and modern interconnect https://en.wikipedia.org/wiki/Xserve https://en.wikipedia.org/wiki/Xserve

zer0zzz 7mo ago

How is this dropping the ball? I think they dropped the ball a long time ago by waiting until M5 to do integrated tensor cores instead of the separate ANE only which was present before. For multi-gpu you can network multiple Macs at high speed now. Their biggest disadvantage to Nvidia right now is that no one wants to do kernel authoring in Metal. AMD learned that the hard way when they gave up on OpenCL and built HIP.

Almondsetat 7mo ago

If Apple doesn't offer a Linux product, they cannot be used seriously in headless computing task. They are adamant in controlling the whole stack, so unless they remake some server version of macOS (and wait years for the community to accustom themselves with it), they will keep being a consumer/professional oriented company

greggsy 7mo ago

They didn’t drop the ball at all? They want to be able to sell handsets, desktops and laptops to their customer base. Pursing a product line that would consume the finite amount of silicon manufacturing resources away from that user base would be corporate suicide. Even nvidia has all but dropped support for its traditional gaming customer base to satisfy its new strategy. At any rate, the local inference capabilities are only going to get cheaper and more accessible over the coming years, and Apple are probably better placed than anyone to make it happen.

gehsty 7mo ago

If my Grandma had wheels she would be a bicycle. Apple would need to transition from being a consumer electronics company to being a B2B retailer for data centre hardware to take advantage of this. Obviously Siri from WWDC 2yrs ago was a disaster for Apple. Other than that they seem to have done pretty well navigating the new LLM world. I do think they would benefit from having their own SOA LLM, but I don’t think its is necessary for them. My mental model for LLMs and Apple is that they are similar Garage Band - “Now everyone can play an instrument” becomes “now anyone can make an app”. Apple owns the interface to the user (i don’t see anyone making nicer to use consumer hardware) and can use what ever stack in the background to deliver the technical features they decide to.

EagnaIonat 7mo ago

> AI training as well as inference Inference has never been an issue for M series, and MLX just ramped it up further. You can do training on the latest MBPs, although any serious models you are going to the cloud anyway.

huslage 7mo ago

Apple makes AI inference and training servers by the thousands. They just don't sell them to anyone. They use them internally in their datacenters. They didn't drop the ball, they are playing a different game while not cannibalizing their existing customer base.

NetMageSCW 7mo ago

Apple didn’t drop the ball - they have no interest in creating servers for a limited time bubble. It is laughable that anyone would think that market would be bigger than iPhone - there’s a reason no RAM manufacturer is building new plants to take advantage of the current demand - they don’t expect it to last long enough to pay for their investment.