6 ms·

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I exp

This seems to be an attempt to compete with people running local models on Apple hardware—even though those local Mac Mini setups aren't really powerful. I expect we'll get there in a few years, so perhaps this is Nvidia taking an early step in that direction. In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding will only increase demand for software engineers, not reduce it. So it also feels like a part of a budding shift in the competitive tension between the various parts of the AI supply chain.

deleted 5mo ago
[deleted]
thewebguyd 5mo ago
Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be betting on that. They don't care where inference happens, so long as it happens on Nvidia hardware.
- CuriouslyC 5mo ago
  I think non-techy users will get subsidized hardware with compute workloads running in the background on idle to recover the cost (and lots of ads).
- h14h 5mo ago
  IMO it's only a matter of time before "self-hosting local AI" is as complicated as installing an app and clicking a download button. And when that happens, the pitch to non-techy users is "Free ChatGPT you can use offline with zero privacy risk". Once hardware accessibility and LLM efficiency advance to the point that this becomes feasible, I suspect it'll result in a much bigger hit to the cloud AI market than many expect.
  - adamrezich 5mo ago
    Why is it only a matter of time? The AI-as-a-service companies are going to continue to improve their products by improving both the part that could be reproduced in a self-hosted setup, but also the “secret sauce” they put on top of that to make it a better product. There is no incentive for this “secret sauce” to be something that can be reproduced for self-hosting, is there?
    - thewebguyd 5mo ago
      What secret sauce? We already have open source tooling for tool use, web browsing, and code execution/computer use. Open weight models will win in the end. AIaaS might keep an edge with multi-modal agentic workflows, but for 80% of general use cases, no "secret sauce" needed, the open weight models are already there, and tooling is constantly getting better. The bottleneck is the cost of local hardware right now.
      - Shitty-kitty 5mo ago
        The "secret sauce" is vendor lock-in. A textbook case is the vmware broadcom situation. Vmware was cheap so corporations found little reason to use open source. Broadcom made vmware expensive but now those corporations are finding out that it is a lot of work (aka expensive) to switch infrastructure.
        woctordho 5mo ago
        The purpose of open source is exactly to fight against vendor lock-in. There is always a way to convert VMs from VMware to open source formats.
    - h14h 5mo ago
      I think a major incentive could be to sell hardware. If Apple is able to get their hands on a local LLM capable of covering a significant % of what people use ChatGPT for, the pitch they can offer is: "Free, private, offline ChatGPT so long as your laptop has X GB of RAM" Beyond that, I wouldn't underestimate the incentive of "because I can". The "secret sauce" you refer to is effectively just a DB & a while loop that feeds text to a bunch of tensors. If an indie dev decides they want to release something that dismantles the OpenAI & Anthropic moats, there really isn't all that big of a technical barrier stopping them.
- artyom 5mo ago
  I'm from the times when you had to purchase a separate chip to perform floating point math. It was called a math co-processor. [1] After a few generations (and over a decade) that was indistinguishable from the CPU chip itself. It's a long hyperbole, I know, but I think local inference is inevitable; and the big fishes know it. Will that be a complex technical setup? An appliance? An additional chip in your motherboard? So transparent it's burned right into the CPU? Those are just implementation details. We're probably just one generational breakthrough away from it. [1] https://en.wikipedia.org/wiki/X87 https://en.wikipedia.org/wiki/X87
  - postalrat 5mo ago
    Like the math co-processor it might end up just being new instructions for the cpu to handle ai related math.
    - woctordho 5mo ago
      And here comes ACE (AI Compute Extensions) on the latest CPUs
- smrtinsert 5mo ago
  > Non-techy consumers may never do it They will. As some point in the future, people will want everything, they'll prompt full movies because they're bored and want to watch something.
  - hdgvhicv 5mo ago
    You’re assuming that owning compute will be possible.
h14h 5mo ago
One can only hope. That said, Apple's vertical integration is a massive competitive advantage here, IMO. Nvidia's reliance on Microsoft & Windows for software support likely makes competing w/ Apple an uphill battle. If/when Local AI gets good enough to compete with Cloud AI on most inference workloads, Apple starts to look like Nvidia's biggest competitor. While this is admittedly a dream scenario, the biggest downside would be Apple effectively having a monopoly in "Agent-ready" consumer electronics. Hopefully local AI both becomes the norm, and there is sufficient competition among the consumer platforms. Side-note: I would love to see an "RTX Spark" Framework 13 mainboard at some point.
- bigyabai 5mo ago
  I don't understand this stance. Microsoft is reliant on Nvidia, they don't have a good ARM SOC to ship with without them. They will bend over backwards to accommodate these SOCs on Windows, and probably don't have much work to do in the first place. Apple's vertical integration has led to a Siri overhaul that took half a decade to roll out, and it won't even run locally. They built an NPU coprocessor that's basically dark silicon for expensive inference, and then shipped MLX to stop Tensorflow and Pytorch from replacing Apple's role in the stack entirely. Mac owners are pleading for signed CUDA drivers for the PCIe or Thunderbolt in their $5,000+ Mac Pros. Apple's ecosystem is pure liability for AI, they're not moving any product for datacenter inference and can't even sell the hardware to themselves: https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-reportedly-sitting-unused-on-warehouse-shelves-due-to-low-apple-intelligence-usage/ https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep... Nvidia's profit margins are safe. Even if the RTX Spark is a completely failed product, Apple is not encroaching on the markets that Nvidia dominates.
  - h14h 5mo ago
    Fair points all around. Ultimately it all comes down to execution. In theory, Apple SHOULD have an advantage given they have everything they need in house and can all pull in a unified direction. In practice, it's not always the case that all the teams in a large corporation are all that much better at pulling in the same direction than multiple different corporations in a partnership. And all this will be moot if Local LLMs never catch up to cloud LLMs in terms of quality. Regardless, it'll be very interesting to see how Nvidia's partnerships with Microsoft & hardware OEMs play out. If the AI inference compute share shifts appreciably to local consumer hardware, I'll want to see strong competition.
bityard 5mo ago
I don't believe Anthropic and OpenAI are any more fearful of local AI than Google or Microsoft are of people hosting their own email. Local AI capabilities are growing at a rapid pace, but so is hosted AI. While you can do a surprising amount of useful work with a model occupying a few to a few hundred gigs of VRAM, the hosted models are going to be way ahead for a long time.
- qdotme 5mo ago
  I fear the same thing, but still am unsure why or how :) Google/Microsoft and hosting your own email is a byproduct of how difficult (socially, not technically) hosting your own email has become - mostly because SMTP protocol is inherently broken by spam and patched by social construct (trusted nodes, abuse@, 3+ DNS entries and counting, etc). Purely technical solutions, such HashCash etc, got discontinued in exchange for social ones. Central providers made (sometimes in exchange for, sometimes as excuse of, spam protection) self-hosting socially hard. Now, I wonder if, and how, once Anthropic and OpenAI need to demonstrate profitability, could hamstring local AI. Which has been /so far/ very valuable for me in doing things that hosted providers don't want liability for, and align against (even if totally lawful and fair use!).
- selicos 5mo ago
  If it's something like: - v4.5: 1x cost, 100% quality, 100% speed but maybe sometimes 80% speed because of load - v4.6: 3x cost, 105% quality, 80% speed most of the time depends - v4.7: 9x cost, 115% quality, 90% speed most of the time Then people will either stick with v4.5 for everything it can do and, if knowledgeable, use v4.7+ for critical or specific tasks. But if we add the option of: LocalLLM: one time hardware + electricity cost, good enough quality for 90% of work, good enough speed for 90% of work, no vendor lock in/sudden cost spikes... Then there is an edge to running it yourself unless you can burn investor cash to get to the next level. I think the recent headlines on org token spend plus my own experience just today (June 1) with the new Copilot Pro limits is going to push those with the compute to run locally. As of about 1pm today I did something to hit 47% of my entire June premium requests (copilot Pro, not converted). As of 2pm I'm using Gemma 4 E4B on a 12gb GPU (with large context window) off my desktop to power VS Code with Copilot on my laptop. I'm going to build an AMD Strix Halo system next week when parts arrive so I can queue up a few models in parallel or work with something I need that much RAM for. I'm not lifting the earth with my LLM setup. Gemma 4 E4B is solid for accelerating my current projects. and it's costing me pennies more per hour vs blowing half my Copilot Pro plan in a distracted morning. I'm at a vendor conference this weekend that is showing off their Agent/Agentic workflows. Nobody can tell me how they balance the cost long term. Hopefully whoever the vendor is paying for their cloud LLM token usage doesn't spike cost in a year (or the vendor themselves) after companies convert and are trapped VMware style with these agent processes. You can bring your own (cloud) model subscription. I need to find out if we can point it back to our own local LLM endpoint and try local models for the same processes. Even if it takes 5x longer, it could be cheaper and more secure.
grahamburger 5mo ago
You can do a lot with existing devices in a medium to decent gaming PC (or probably phone/laptop, I haven't tried.) I think HN tends to skew toward only thinking of LLM as useful for coding, but they are very useful for many non-coding things, and existing local LLMs are quite capable. I imagine it won't be long before apps with LLM-based features will try to run locally first and fall back to cloud LLMs just to save token costs. Actually I'd be surprised if some apps aren't doing this already.
spamizbad 5mo ago
Might be aimed at people who spec out the $5100 Macbook Pros with M5 Maxes and 128GB.
- spullara 5mo ago
  definitely! it has the advantage that it can run CUDA kernels but on the other hand it has lower memory bandwidth and probably loses a token/s fight for many LLMs.
mschuster91 5mo ago
> In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding will only increase demand for software engineers, not reduce it. The writing is on the wall, neither Anthropic nor OpenAI are anywhere near close to sustainability and if one or, worse, both fail the entire demand bubble for NVDA crashes. It's smart to set up alternative destination markets while they can do so in peace.
c7b 5mo ago
It's not even anything new, it's basically the mobile version of the DGX Spark. The two chips (N1X/GB10) are pretty similar in terms of architecture and specs. I don't get why this seems to be getting so much attention now. But I like it. It's a copy of Apple's SoC design philosophy, same as AMD's Strix Halo, which I always thought was really cool both for laptops and home PCs. NVidia's traditional consumer cards pull way too much power and are too noisy to comfortably put them in a living or office environment.

deleted 5mo ago

[deleted]

thewebguyd 5mo ago

Local AI was/is bound to happen, eventually. It'd be smart of Nvidia to get ahead of it. Non-techy consumers may never do it, but at some point businesses are going to start asking when do they stop paying per token and start running models themselves. Right now the hardware is cost prohibitive, but I doubt that'll always be the case. Eventually the hardware will get cheaper and more available, and Nvidia seems to be betting on that. They don't care where inference happens, so long as it happens on Nvidia hardware.

CuriouslyC 5mo ago
I think non-techy users will get subsidized hardware with compute workloads running in the background on idle to recover the cost (and lots of ads).
h14h 5mo ago
IMO it's only a matter of time before "self-hosting local AI" is as complicated as installing an app and clicking a download button. And when that happens, the pitch to non-techy users is "Free ChatGPT you can use offline with zero privacy risk". Once hardware accessibility and LLM efficiency advance to the point that this becomes feasible, I suspect it'll result in a much bigger hit to the cloud AI market than many expect.
- adamrezich 5mo ago
  Why is it only a matter of time? The AI-as-a-service companies are going to continue to improve their products by improving both the part that could be reproduced in a self-hosted setup, but also the “secret sauce” they put on top of that to make it a better product. There is no incentive for this “secret sauce” to be something that can be reproduced for self-hosting, is there?
  - thewebguyd 5mo ago
    What secret sauce? We already have open source tooling for tool use, web browsing, and code execution/computer use. Open weight models will win in the end. AIaaS might keep an edge with multi-modal agentic workflows, but for 80% of general use cases, no "secret sauce" needed, the open weight models are already there, and tooling is constantly getting better. The bottleneck is the cost of local hardware right now.
    - Shitty-kitty 5mo ago
      The "secret sauce" is vendor lock-in. A textbook case is the vmware broadcom situation. Vmware was cheap so corporations found little reason to use open source. Broadcom made vmware expensive but now those corporations are finding out that it is a lot of work (aka expensive) to switch infrastructure.
      - woctordho 5mo ago
        The purpose of open source is exactly to fight against vendor lock-in. There is always a way to convert VMs from VMware to open source formats.
  - h14h 5mo ago
    I think a major incentive could be to sell hardware. If Apple is able to get their hands on a local LLM capable of covering a significant % of what people use ChatGPT for, the pitch they can offer is: "Free, private, offline ChatGPT so long as your laptop has X GB of RAM" Beyond that, I wouldn't underestimate the incentive of "because I can". The "secret sauce" you refer to is effectively just a DB & a while loop that feeds text to a bunch of tensors. If an indie dev decides they want to release something that dismantles the OpenAI & Anthropic moats, there really isn't all that big of a technical barrier stopping them.
artyom 5mo ago
I'm from the times when you had to purchase a separate chip to perform floating point math. It was called a math co-processor. [1] After a few generations (and over a decade) that was indistinguishable from the CPU chip itself. It's a long hyperbole, I know, but I think local inference is inevitable; and the big fishes know it. Will that be a complex technical setup? An appliance? An additional chip in your motherboard? So transparent it's burned right into the CPU? Those are just implementation details. We're probably just one generational breakthrough away from it. [1] https://en.wikipedia.org/wiki/X87 https://en.wikipedia.org/wiki/X87
- postalrat 5mo ago
  Like the math co-processor it might end up just being new instructions for the cpu to handle ai related math.
  - woctordho 5mo ago
    And here comes ACE (AI Compute Extensions) on the latest CPUs
smrtinsert 5mo ago
> Non-techy consumers may never do it They will. As some point in the future, people will want everything, they'll prompt full movies because they're bored and want to watch something.
- hdgvhicv 5mo ago
  You’re assuming that owning compute will be possible.

h14h 5mo ago

One can only hope. That said, Apple's vertical integration is a massive competitive advantage here, IMO. Nvidia's reliance on Microsoft & Windows for software support likely makes competing w/ Apple an uphill battle. If/when Local AI gets good enough to compete with Cloud AI on most inference workloads, Apple starts to look like Nvidia's biggest competitor. While this is admittedly a dream scenario, the biggest downside would be Apple effectively having a monopoly in "Agent-ready" consumer electronics. Hopefully local AI both becomes the norm, and there is sufficient competition among the consumer platforms. Side-note: I would love to see an "RTX Spark" Framework 13 mainboard at some point.

bigyabai 5mo ago
I don't understand this stance. Microsoft is reliant on Nvidia, they don't have a good ARM SOC to ship with without them. They will bend over backwards to accommodate these SOCs on Windows, and probably don't have much work to do in the first place. Apple's vertical integration has led to a Siri overhaul that took half a decade to roll out, and it won't even run locally. They built an NPU coprocessor that's basically dark silicon for expensive inference, and then shipped MLX to stop Tensorflow and Pytorch from replacing Apple's role in the stack entirely. Mac owners are pleading for signed CUDA drivers for the PCIe or Thunderbolt in their $5,000+ Mac Pros. Apple's ecosystem is pure liability for AI, they're not moving any product for datacenter inference and can't even sell the hardware to themselves: https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-reportedly-sitting-unused-on-warehouse-shelves-due-to-low-apple-intelligence-usage/ https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep... Nvidia's profit margins are safe. Even if the RTX Spark is a completely failed product, Apple is not encroaching on the markets that Nvidia dominates.
- h14h 5mo ago
  Fair points all around. Ultimately it all comes down to execution. In theory, Apple SHOULD have an advantage given they have everything they need in house and can all pull in a unified direction. In practice, it's not always the case that all the teams in a large corporation are all that much better at pulling in the same direction than multiple different corporations in a partnership. And all this will be moot if Local LLMs never catch up to cloud LLMs in terms of quality. Regardless, it'll be very interesting to see how Nvidia's partnerships with Microsoft & hardware OEMs play out. If the AI inference compute share shifts appreciably to local consumer hardware, I'll want to see strong competition.

bityard 5mo ago

I don't believe Anthropic and OpenAI are any more fearful of local AI than Google or Microsoft are of people hosting their own email. Local AI capabilities are growing at a rapid pace, but so is hosted AI. While you can do a surprising amount of useful work with a model occupying a few to a few hundred gigs of VRAM, the hosted models are going to be way ahead for a long time.

qdotme 5mo ago
I fear the same thing, but still am unsure why or how :) Google/Microsoft and hosting your own email is a byproduct of how difficult (socially, not technically) hosting your own email has become - mostly because SMTP protocol is inherently broken by spam and patched by social construct (trusted nodes, abuse@, 3+ DNS entries and counting, etc). Purely technical solutions, such HashCash etc, got discontinued in exchange for social ones. Central providers made (sometimes in exchange for, sometimes as excuse of, spam protection) self-hosting socially hard. Now, I wonder if, and how, once Anthropic and OpenAI need to demonstrate profitability, could hamstring local AI. Which has been /so far/ very valuable for me in doing things that hosted providers don't want liability for, and align against (even if totally lawful and fair use!).
selicos 5mo ago
If it's something like: - v4.5: 1x cost, 100% quality, 100% speed but maybe sometimes 80% speed because of load - v4.6: 3x cost, 105% quality, 80% speed most of the time depends - v4.7: 9x cost, 115% quality, 90% speed most of the time Then people will either stick with v4.5 for everything it can do and, if knowledgeable, use v4.7+ for critical or specific tasks. But if we add the option of: LocalLLM: one time hardware + electricity cost, good enough quality for 90% of work, good enough speed for 90% of work, no vendor lock in/sudden cost spikes... Then there is an edge to running it yourself unless you can burn investor cash to get to the next level. I think the recent headlines on org token spend plus my own experience just today (June 1) with the new Copilot Pro limits is going to push those with the compute to run locally. As of about 1pm today I did something to hit 47% of my entire June premium requests (copilot Pro, not converted). As of 2pm I'm using Gemma 4 E4B on a 12gb GPU (with large context window) off my desktop to power VS Code with Copilot on my laptop. I'm going to build an AMD Strix Halo system next week when parts arrive so I can queue up a few models in parallel or work with something I need that much RAM for. I'm not lifting the earth with my LLM setup. Gemma 4 E4B is solid for accelerating my current projects. and it's costing me pennies more per hour vs blowing half my Copilot Pro plan in a distracted morning. I'm at a vendor conference this weekend that is showing off their Agent/Agentic workflows. Nobody can tell me how they balance the cost long term. Hopefully whoever the vendor is paying for their cloud LLM token usage doesn't spike cost in a year (or the vendor themselves) after companies convert and are trapped VMware style with these agent processes. You can bring your own (cloud) model subscription. I need to find out if we can point it back to our own local LLM endpoint and try local models for the same processes. Even if it takes 5x longer, it could be cheaper and more secure.

grahamburger 5mo ago

You can do a lot with existing devices in a medium to decent gaming PC (or probably phone/laptop, I haven't tried.) I think HN tends to skew toward only thinking of LLM as useful for coding, but they are very useful for many non-coding things, and existing local LLMs are quite capable. I imagine it won't be long before apps with LLM-based features will try to run locally first and fall back to cloud LLMs just to save token costs. Actually I'd be surprised if some apps aren't doing this already.

spamizbad 5mo ago

Might be aimed at people who spec out the $5100 Macbook Pros with M5 Maxes and 128GB.

spullara 5mo ago
definitely! it has the advantage that it can run CUDA kernels but on the other hand it has lower memory bandwidth and probably loses a token/s fight for many LLMs.

mschuster91 5mo ago

> In that case, this goes against Anthropic and OpenAI's business models. Which is a double whammy after Jensen Huang's recent comment about how agentic coding will only increase demand for software engineers, not reduce it. The writing is on the wall, neither Anthropic nor OpenAI are anywhere near close to sustainability and if one or, worse, both fail the entire demand bubble for NVDA crashes. It's smart to set up alternative destination markets while they can do so in peace.

c7b 5mo ago

It's not even anything new, it's basically the mobile version of the DGX Spark. The two chips (N1X/GB10) are pretty similar in terms of architecture and specs. I don't get why this seems to be getting so much attention now. But I like it. It's a copy of Apple's SoC design philosophy, same as AMD's Strix Halo, which I always thought was really cool both for laptops and home PCs. NVidia's traditional consumer cards pull way too much power and are too noisy to comfortably put them in a living or office environment.