13 ms·
Nvidia is proposing a beast of a CPU system for Windows PCs
- Waterluvian 5mo agoIt’s an opportunity for them to start doing away with the whole ATX thing where owners had freedom to mix and match at their own pleasure.
- burnt-resistor 5mo agoThey'll ship a welded-shut box that requires an activation key to power on. Users will get to pick color sleeve it uses though.
- sherazp995 5mo agoWait a minute! Nvidia going from GPU to CPU now?
- wmf 5mo agoNvidia has been making CPUs for 10-15 years.
- userbinator 5mo agoMuch longer than that... https://theretroweb.com/chipsets/182 https://theretroweb.com/chipsets/182 https://www.nvidia.com/en-us/drivers/uli-m6117c/ https://www.nvidia.com/en-us/drivers/uli-m6117c/
- TiredOfLife 5mo agoBut this one has mediatek cpu with off the shelf arm cores
- Aperocky 5mo agoWhy is it only for Windows PC, can we not run Linux or at minimum SteamOS?
- cyberziko 5mo agogood to know, hope the price will be affordable, having a pc becoming a luxury :)
- crims0n 5mo agoCertainly not in the year of our lord, 2026. Maybe in a few years though.
- deleted 5mo ago[deleted]
- dgellow 5mo agoI’m not sure if you’re aware but there is a supply chain shortage for pretty much everything needed for a PC that isn’t expected to be solved this year or next year. There is no way that can be affordable
- 8note 5mo agoor moreso, the available supply has been eaten by rampant speculation, and hyperscalers have overpurchased vs the datacenters they can actually get built and power
- YasuoTanaka 5mo ago128GB of unified memory is a dream come true for local LLMs. VRAM has been the ultimate bottleneck for developers.
- avocadoking 5mo agoIt could help with exploding external LLM costs. Interesting to see how the adaption will be, which will mainly depend on the price.
- zackify 5mo ago[dead]
- adrian_b 5mo agoThe competitor for this NVIDIA CPU will not be the now old AMD Strix Halo, but its successor (launched recently), which supports up to 192 GB of unified memory. Thus 128 GB is no longer SOTA. While this NVIDIA system is inferior from the point of view of the memory capacity, its main advantage is that the top models will have a bigger GPU, i.e. with 6144 or 5120 FP32 execution units, compared to 2560 for the AMD GPU (compared to the NVIDIA CPU, the AMD CPU has a better multi-threaded performance for legacy programs, and a much better multi-threaded performance for the applications that use AVX-512). However, these top models with big GPUs will also be much more expensive than the competing AMD system, while also being much more expensive than a laptop or mini-PC with an equivalent discrete NVIDIA GPU (which has the disadvantage of having direct access only to a much smaller, even if faster, memory).
- christkv 5mo agoI don’t think there is much improvement in compute for the new strix halo revision. The next one supposedly adds rdna4 cores or similar and more memory channels
- adrian_b 5mo agoThere is no improvement in the CPU or GPU, except for minor increases in the clock frequency. The memory interface is a little faster, but the greatest improvement is +50% in the memory capacity, both over the old Strix Halo and over NVIDIA Spark. However, even the Strix Halo CPU was better than the NVIDIA/Mediatek CPU. NVIDIA has only the advantage (in its more expensive variants) of a GPU equivalent with RTX 5070. It remains to see which will be the prices of the NVIDIA Spark models with big GPUs, but the rumors are that they grow from around $3000 upwards, with the upper limit for 128 GB DRAM and uncut GPU being unknown yet. It also remains to be seen whether the variants with the biggest GPU can use it effectively when having a rather low memory bandwidth for such a big GPU.
- jqpabc123 5mo agoI am not sure how many people will run AI models locally. It still seems like a niche application to me. I'd say this relates directly to the cost of running AI models remotely. And we won't know what the actual cost will be until AI vendors recover the huge pile of cash they've dumped into development (plus interest).
- chpatrick 5mo agoI think it's niche now because getting the hardware to run it is expensive and the quantized models don't work as well. If those improve then it would be a no brainer to pay one off for the hardware instead of a fortune for API calls.
- jqpabc123 5mo agoAI vendors are attempting to offer the whole apple. And they are spending huge sums of money in the process. But most businesses don't really care about most of the apple --- they only need their special bite out of it. For example, doctors mainly care about medicine. Nvidia is attempting to provide the hardware needed for local, specialized models.
- dofm 5mo agoI think it is likely to appeal to video and photo editors who want to use AI tools (the press release has a quote from Blackmagic Design, as well as from Adobe, who I think have no stomach for their own cloud AI). But I don’t know about specialised: this could run quite large models with MoE.
- dofm 5mo agoI am not really convinced that four bit quantisation is that bad; almost certainly six will be enough. But Google are making claims for their QAT tech in Gemma that they are surely using or testing in Gemini that it preserves nearly source model quality while reducing footprint. The hardware for 50 tokens per second with a four bit quantisation of Gemma 4 26B or the sparse Qwen 3.6 is not really that expensive: it’s a secondhand M1 Max. Beyond that, I agree. I think moving planning tasks to local is a now thing, not that it really has much impact on token spend. I also think many small coding tasks are fully within the grasp of the above two models. The main issue right now is that the software landscape is rather confusing, but I reckon uncomplicated Gemma 4 26B QAT support with MTP is a few weeks away.
- tosh 5mo agonb: poster is Daniel Lemire (https://lemire.me https://lemire.me), who is very skilled in getting performance out of compute hardware (e.g. via simd, cache usage etc)
- infecto 5mo agoAs he likes to share often, "He ranks among the top 2% of scientists globally (Stanford/Elsevier 2025) and is one of GitHub's top 1000 most followed developers. "
- tempodox 5mo agoStill, Microslop has repeatedly proven their ability to slow everything down to a crawl no matter how powerful the hardware. If you want it to be fast, don’t use Windows.
- 2OEH8eoCRo0 5mo agoAre their enterprise orders slowing down? Why use precious maxed out fab capacity on consumer stuff when it could be an enterprise chip?
- zamadatix 5mo agoIt uses LPDDR5X instead of VRAM and will still sell for a premium while pushing their presence even further in every side of the AI market. This was one area AMD was ahead in and now Nvidia is probably better off making this to compete on that front while still being better off than making a 5090.
- fc417fc802 5mo agoThat doesn't answer the question. If the high margin enterprise GPUs are saturating the fab capacity you wouldn't expect them to be pushing this. But IIRC those all have oodles of integrated HBM at this point so I wonder if fab capacity for that has become a bottleneck.
- deleted 5mo ago[deleted]
- zamadatix 5mo agoI believe it does - the reasons why are exactly differences like LPDDR5X vs HBM3e. Not every fab is capable of making any type of chip another fab makes. If you can make a product with different chips and still sell it for a premium why would you not just because the fabs for your DC product's chips are busy? Looking at it more, I believe the story repeats with the TSMC processes used for the CPU vs chips like GB200 as well. Even if none of the above were the case, the question still isn't "why not make the enterprise GPU" it's "why not make the higher margin per chip area product". If the NV1/GB10 take less die space and cost a lot it's not immediately apparent the enterprise GPU actually nets Nvidia more $ per die or not. That's why it's relevant these will be sold at a premium.
- dofm 5mo agoIt already is an enterprise chip. This is about Microsoft not having the equivalent of an M3 Max or whatever laptop. And maybe for NVIDIA and MS it is also about them quietly betting that local models are, in fact, going to be good enough for most tasks pretty soon.
- llm_nerd 5mo agoDoes this person know that this is the same GB chip in the DGX Spark? It isn't some proposed thing, it's a chip loads of people have on their desk right now, and there are endless benchmarks of it. Decent single core (a long ways from Apple level, but decent), but it makes up for it in cores to provide M5 level performance, CPU wise. Memory bandwidth it is kind of starved, at 1/6th many GPUs. They got Microsoft to customize Windows for the RTX Spark, and will likely have to brutally throttle it when running as a laptop (it's literally a 140W TDP chip), and that's neat. It's going to be a very expensive laptop.
- Apreche 5mo agoI heard the memory bandwidth is not just slower than on a GPU, as expected, but is significantly slower than Apple’s unified memory.
- MrBuddyCasino 5mo agoCPU/GPU is decent (800 GB or so), memory is slowish (300GB or so). Some Apple M are slower, some are faster.
- dagmx 5mo agoWhere did you get those numbers from? DGX Spark has a maximum of 273 GB/s bandwidth in ideal scenarios (hard to reach) That puts it between an M5 (153) and M5 Pro (307)
- MrBuddyCasino 5mo agoThe 900 GB/s is from the NVLink-C2C interconnect, if you were wondering about that. They quote "up to 900 GB/s of bidirectional bandwidth between GPU and CPU". Mind you thats not to/from memory, which indeed only has 273 GB/s.
- dagmx 5mo agoAh I see. But the only C2C equivalent on the Apple side is the UltraFusion which is 2.5TB/s if I recall correctly.
- seanalltogether 5mo agoIs it really unified memory? AMD Strix Halo is "unified" but you still have to allocate memory separately for cpu vs gpu. Apple Silicon is true unified memory.
- joe_mamba 5mo ago>AMD Strix Halo is "unified" but you still have to allocate memory separately for cpu vs gpu. IIRC that's due to maintain BIOS and Windows (+games & apps) backwards compatibility, but memory access speeds are the same.
- ankurdhama 5mo agoIt is unified in the sense that the OS can dynamically assign memory to CPU and GPU. Apple silicon is not a alien tech that other silicon vendors cannot implement.
- ApatheticCosmos 5mo agoStrix halo is unified memory. The memory allocation set in BIOS is overridden by the operating system if it has the capability.
- eigenspace 5mo agoThat's a software question, not a hardware question. Some software assumes pre-defined set-aside pools of memory reserved for video purposes, but the chip does actually have access to the whole pool.
- fc417fc802 5mo ago> you still have to allocate memory separately for cpu vs gpu That's an API issue not a hardware issue. Regardless, I believe the major APIs permit seamlessly sharing pointers at this point? (I have no experience doing that though.)
- flakiness 5mo agoMy understanding is that this is the limitation from Windows not from AMD SoC. There are several internet resources to "enable unified memory support" on linux eg [1]. As a side note, qualcomm chip set on Android has been doing this for years (like Apple) so it's not super unique thing. It's more like there was no need before. [1] https://www.jeffgeerling.com/blog/2025/increasing-vram-allocation-on-amd-ai-apus-under-linux/ https://www.jeffgeerling.com/blog/2025/increasing-vram-alloc...
- sisve 5mo ago> I am not sure how many people will run AI models locally. It still seems like a niche application to me. Bill Gates had a quote some years ago... People have still not learned how fast we improve our tech and how much cheaper thing gets I guess :)
- chaostheory 5mo agoWe had a thing called globalism that drastically reduced costs. Globalism right now is on life support. Given geopolitics, I don’t see how it’s going to survive.
- sisve 5mo agoYou are right. I haven't considered that enough. I do agree with you that globalism will be decreased even in the context of a decade, but I think it will be on a level that still give us some cost reduction. But I do think (at least hope) geopolitics will be better in 5-7 years.
- dgellow 5mo agoMemory isn’t getting cheap soon, and you need a lot of it for local models
- sisve 5mo agoAll depends. The current technology will be cheaper in a year or two. The best cutting edge stuff will properly be even more expensive. But in 10 years time... we can run current SOTA models (or models that are equally good ) on our local hardware
- dgellow 5mo agoAh yes, if you count in decades, for sure I expect to run them locally
- infecto 5mo ago"I am not sure how many people will run AI models locally. It still seems like a niche application to me. However, it will make decent machines to play video games." I don't know who will be the winner but with some of the recent releases from gemma it seems more probable that you may run some models locally if only from a cost perspective, not even considering business security. Not sure how this type of architecture would make for good gaming though, puts into question the whole statement. "Ranked in the top 2% of scientists globally (Stanford/Elsevier 2025) and among GitHub's top 1000 developers" - side note but this guy puts this everywhere, gives me probably the inverse of what he is marketing for.
- iLoveOncall 5mo ago> "Ranked in the top 2% of scientists globally (Stanford/Elsevier 2025) and among GitHub's top 1000 developers" - side note but this guy puts this everywhere, gives me probably the inverse of what he is marketing for. Lol yeah seriously, that stinks "I ask AI to generate a huge amount of bullshit and upload it to pad irrelevant stats". Absolute loser.
- deleted 5mo ago[deleted]
- netsharc 5mo agoI found his website, https://www.lemire.me/en/ https://www.lemire.me/en/ , and the "2%" brag is the very first sentence, geez. Being the top x% is what OnlyFans girls brag about, professor... And it's not exactly brain surgery, is it? https://www.youtube.com/watch?v=THNPmhBl-8I https://www.youtube.com/watch?v=THNPmhBl-8I
- Zetaphor 5mo ago> Daniel Lemire’s blog is one of the top 50 most popular blogs on Hacker News, the standard tech news aggregation site. Citation needed
- 5mo ago
- alberth 5mo agoIs this essentially an Apple M-Series chip in concept?
- trynumber9 5mo agoNo, it stems from a lineage of Tegra chips that pre-date the M-Series. This chip was called GB10. One of its predecessors, GV10 was shipped in 2018. It was a 256 bit, unified-memory system on a chip with a Volta GPU and 12 ARM Cores. GB10 is a 256 bit, unified-memory system on a chip with a Blackwell GPU and 10+10 ARM cores.
- BoredPositron 5mo agoMediatek and Nvidia the horsemen of abandoning hardware after a year. The Jetson family still left a bad taste in my mouth.
- burnt-resistor 5mo agoHow dare you question the golden goose egg-laying algorithm for trillions in stock valuation!
- thewebguyd 5mo agoQualcomm is too. They mainlined the GPU firmware for the X Elite 2nd gen, but still have not done so for their 1st gen X Elite which they promised full Linux support for and failed to deliver, and have now moved on pretending they never said that.
- deleted 5mo ago[deleted]
- SwtCyber 5mo agoThe interesting part to me isn't really the Cortex-X925 vs AVX-512 comparison, but Nvidia trying to make the GPU the center of a Windows PC rather than an add-in card
- deleted 5mo ago[deleted]
- QQ00 5mo ago>but Nvidia trying to make the GPU the center of a Windows PC rather than an add-in card over the last decade, many software (especially the popular and industry standard ones) shifted to GPU accelerated design. it's a push before NVIDIA even tried to capitalize on that.
- cryo32 5mo agoYeah when laptops are shipping 8Gb and Microsoft is suddenly interested in native apps, nope. Tech companies have strangled their own market.
- thewebguyd 5mo agoLaptops shipping with less RAM is exactly the reason to be interested in native apps again. Every app being a chrome/EdgeWebView process is the problem.
- AmazingTurtle 5mo agowhile unified memory may offer better performance than unsoldered DDR system memory, it still won't be as great as 1.8TB/s bandwidth on high end consumer GPUs right now. nvidias master plan may be making it the new normal to have "only" 400GB/s bandwidth, thus gatekeeping local model usage further behind "more memory but not as fast as the cloud can do it"
- dangus 5mo agoI think it’s an interesting theory but a bit too conspiracy theory-ish. Nvidia just wants to sell stuff to everyone. And I think for professionals doing local AI work, products like Strix Halo and Apple Silicon are a competitive threat. A big part of maintaining the leading software ecosystem is ensuring you have competitive hardware for all your users. I also think the RTX Spark product is relatively low effort for Nvidia. Grab a Mediatek CPU and slap an Nvidia GPU on the die. Sure, that’s oversimplifying it, but still.
- dofm 5mo agoHere is the press release for the actual machine: https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-pcs-agents-rtx-spark https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-... I have been somewhat surprised at the lack of commentators observing that this is Microsoft and above all NVIDIA launching a device that is fundamentally at odds with the metered cloud model of AI. When you look at the other announcements and murmurings (better offline BYOK for Copilot, talk of an unmetered AI future) I think it’s clear that these two firms understand that cloud-only AI is not sustainable or inherently in their interests. But their willingness to undermine OpenAI with a product like this is notable.
- tantalor 5mo agoMaybe. Or they are simply hedging their bets.
- gscott 5mo agoThey get to keep more Ai usage from big providers, send telemetry, and inject ads.
- thewebguyd 5mo agoYeah, "unmetered intelligence" was probably the most used phrase at MS BUILD this past week. They are going hard into local AI
- dofm 5mo agoI don't think you can interpret this as anything other than a sanctioned rebuke, right? Everyone has a strong visceral sense for what that means. Copilot just got proper "offline" BYOK support, didn't it? Presumably that was one of the things they were talking about. Though I imagine that has something to do with the fact that Zed has supported that properly for months.
- Yokohiii 5mo agoFor me this is a push to segment the market into consumer and industrial grade RAM. Even NVIDIA and MS are not stupid enough to think they can keep going with RAM prices exploding. Consumers need hardware to subscribe to their AI stuff. LLMs will get bigger and even with 128GB (that many wont saturate), you wont run future frontier models. For LLM vendors and integrators it's a handy thing to move lower quality inference to the consumers. Also running local doesn't have to mean that the models have open weights. MS will likely start to distribute closed models at scale once the hardware is there.
- thrance 5mo agoWill it support Linux?
- ChrisArchitect 5mo agoRelated: A powerful new chapter for Windows PCs, accelerated by Nvidia RTX Spark https://news.ycombinator.com/item?id=48352693 https://news.ycombinator.com/item?id=48352693 Nvidia RTX Spark https://news.ycombinator.com/item?id=48352939 https://news.ycombinator.com/item?id=48352939
- PedroBatista 5mo agoDon't want to be too harsh, maybe I'm missing something, but the CPU is at least 2 years old, internally it has been a complete shitshow and that's a minor hiccup when compared to the firmware and software situation. It's an interesting "newcomer" and the more the better but calling this a "beast" and a "game changer" is ridiculous to say the least. Then there is the price..
- dagmx 5mo agoThis feels fluff to me on the part of the author (whose work I don’t want to trivialize) but I don’t think they’ve actually looked deeper than a paper spec sheet on this. 1. Yes it has the same number of cores as a 5070 mobile. It’s also running at a shared peak of 2/3 the bandwidth and a shared peak of 2/3 the TDP. The GPU by itself will likely perform at half the dedicated units performance 2. Apple may not have SVE2 but they do have the AMX (private) and SME. I don’t see why he thinks the SVE2 will give him more performance than the SME. 3. He mentions a single core type but doesn’t mention the total makeup. We already have known for a year how the DGX Spark compares to Apple chips. For CPU it’s roughly equivalent to an M3 Pro and for GPU compute (not rasterization) it’s between an M4 Pro and M4 Max without considering bandwidth. The real advantage to these is that they run CUDA. That’s it. Otherwise when they launch they’ll be 2-3 generations behind where Apple is and 1 gen behind AMD. The other super power of the DGX Spark was the NIC for pairing them together. But that’s been removed here too.
- llm_nerd 5mo agoIt is absolutely fluff, and the only reason this worthless tweet is on the front page of HN is that this audience has a habit of canonizing certain people, and treating each of their bowel movements as prophetic. Guy suddenly became aware of a chip that the rest of the industry long knew about, seems completely unaware of the competitors, and posts about how it's a BEAST and will be a GAME CHANGER. Like the DGX Spark was a game changer? Eh, it has mostly been a massive disappointment. An overpriced nvidia laptop isn't going to change the equation an iota.
- trympet 5mo agoYes. This reads like a LinkedIn post rehashing old news. I’m not even in the industry.
- storus 5mo ago> GPU compute (not rasterization) it’s between an M4 Pro and M4 Max without considering bandwidth You are likely thinking about token generation which is dependent on memory bandwidth where Apple has an edge. Spark's GPU compute is way higher than even M5 Max (17 FP32 TFlops), around 2x FP32 TFlops... It's literally 6144 CUDA cores like desktop 5070, slowed down by slow memory and lower TDP (29.7 vs 31 FP32 TFlops on 5070).
- theturtle 5mo ago[dead]
- GuestFAUniverse 5mo agoAnd who in 2026 is still anal-fixated on a "Windows" PC? It's just a personal computer. It normally runs multiple operating systems just fine. Windows PC sounds like people talking about tech who are either payed by M$, or embed pictures into Word documents to send them. Nobody has to kill the fun those OS agnostic machine allow, by artificially bind them to a shitty OS.
- zdragnar 5mo agoEnterprise, of course. They probably buy more PCs than the rest of the market combined. Even for personal use, I'd imagine the amount of people dual booting Windows and something else are a very tiny minority. Saying "Windows PC" is a pretty reasonable way to distinguish between "made by Apple" and "made by someone else" because the market of PCs that aren't made by Apple and don't come with Windows is really, really tiny. To be honest, this seems like a strange hill to take such an aggressive stance upon.
- bigyabai 5mo ago> It's just a personal computer Your x86 machines were, but these are ARM SOCs. Many of them don't even support UEFI, let alone the upstream Linux kernel.
- rvba 5mo agoGetting rid of UEFI is bad?
- bigyabai 5mo agoCan you quote where I said that?
- dylan604 5mo agoSir! I'm not an LLM.
- speed_spread 5mo ago
- sometimelurker 5mo agocant wait til someone figures how to run Linux on one of these
- easygenes 5mo agoThat happened a year ago when these shipped as the DGX Spark with only Linux pre installed.
- embedding-shape 5mo ago> up to 6,144 state-of-the-art CUDA cores A RTX Pro 6000 has ~24K 5th generation tensor cores, I'm guessing this would then be 1/4 of the count but 6th generation? Wasn't clear from the images.
- gravypod 5mo agoWhat is more important than core count is how the caching architecture is laid out. They could lay out those 6k cuda cores in a layout which provides much larger blocks of cache to smaller number of cores. That would increase the memory bandwidth which would be better for inference.
- embedding-shape 5mo agoSounds like the memory bandwidth is worse though; > The memory is not as fast as dedicated GPU memory, but it is cheap enough while delivering enough bandwidth to run AI models locally. Also "cheap while delivering enough" certainly sounds like someone is trying to temper expectations. It sounds like something sitting in-between GPU+VRAM inference and CPU+RAM one, not as a step above/besides GPU+VRAM.
- gravypod 5mo agoHaving slower memory may not actually lead to lower memory bandwidth. The cuda cores can be broken up into compute complexes which larger blocks of memory directly attached to the cores. These could be filled with read operations from the bulk system memory. You can start executing and then page the next batch of data in while compute is working. For LLMs you don't have much random memory access, you can sequence your accesses in blocks. If these chips become popular I am sure you will see LLM architectures taking advantage of the parallelism.
- cthalupa 5mo ago> The cuda cores can be broken up into compute complexes which larger blocks of memory directly attached to the cores. Perhaps in theory, but for the gb10 stuff the memory is all on the CPU die and connected to the GPU die via nvlink-c2c
- npn 5mo agoIs this somehow satire? This is just the dgx spark with keyboard and monitor in a convenient format. Since it has more stuff, I'm sure that the price mark up will increase too. Up to $5000 because why not? With that money you can build a real PC with rtx 5090!
- thewebguyd 5mo agoNot with 128GB (less OS) available to the GPU you can't. The unified memory is the point with this machine (and the dgx spark).
- ozgrakkurt 5mo agoSays running local llms isn’t relevant. Than says it is decent for games, which is just correct if you compare any gpu remotely similarly priced. I don’t understand what is the point he is making
- jmyeet 5mo agoThis is the RTX Spark [1]. The obvious comparison here is the M5 Max where you can buy a Macbook Pro with 128GB of also unified memory. Obviously CUDA cores are specific to NVidia so it's hard to directly compare but I've seen claims that the M5 Max is roughly equivalent to ~4000 CUDA cores. This obviously depends on workload and whether the CPU supports the precision you want to use (eg FP4). The M5 Max has memory bandwidth of 819GB/s. The RTX Spark I believe is ~600. So it might be slightly better than the current generation of Macs but likely worse than the expected M5 Ultras of the new Mac Studios (likely Q3 2026). For comparison, a 5090 has >20k CUDA cores and 1800GB/s memory bandwidth with 32GB of VRAM. The RTX 6000 Pro (at ~$10k) has 96GB of VRAM, same bandwidth and ~24k CUDA cores. We have to see what RTX Spark systems sell for but the DGX Spark is in the Mac Studio price range (~$4k). I do think Apple has a real opportunity here but there offerings aren't quite there yet. The M5 Ultras might be a really attractive option for local LLMs. I expect them to be in high demand. [1]: https://news.ycombinator.com/item?id=48352939 https://news.ycombinator.com/item?id=48352939
- bigyabai 5mo ago> I've seen claims that the M5 Max is roughly equivalent to ~4000 CUDA cores Who claimed that? The M5 is still a raster focused GPU, dedicated matmul blocks be damned. For some workloads that napkin math might work out, but for many others it's a wild overshoot. Time-to-first-token still favors CUDA, and real-world training workloads aren't getting anywhere near Apple Silicon. All of the memory bandwidth in the world is useless if you spend 15 minutes processing 64k tokens worth of context prefill. This is where CUDA shines.
- modeless 5mo agoThe Qualcomm Snapdragon X2 Elite Extreme trounces Nvidia's chip in single core CPU performance. It beats Intel and AMD's best, too. It has unified memory. It's the only CPU in the same league as Apple's M-series in both CPU performance and power efficiency. And it's available in laptops today, not later this year. People are sleeping on Qualcomm.
- bradfa 5mo agoQualcomm is a “fool me once, shame on you, fool me twice, you don’t fool me twice” kind of situation. So many horrible experiences in the past that people are going to be hesitant. Qualcomm are trying harder now it seems. But it will take time to repair their reputation in the PC market.
- derefr 5mo agoCan you say more? I don't have any memory of Qualcomm-related scandals(?), but I just read the news; I've never really been a user of their chips.
- thewebguyd 5mo agoThey burned me with the first gen Snapdragon X Elite. Before the various laptops with it were out they promised Linux support. Here we are, years alter, still no fully OOTB support. Ironically, the GPU firmware were just mainlined in the kernel 4 months ago, but they still haven't done the same for the 1st gen X elite. Tuxedo computers tried and didn't succeed either. I will never buy Qualcomm again. I avoid them on phones as well by just buying Apple. They do not support their hardware beyond the release.
- jeroenhd 5mo ago> I avoid them on phones as well by just buying Apple To each their own, but I don't recall Apple ever mainlining any of their drivers on Linux. You're rightfully angry on the laptop side of things, but Apple is much worse than Qualcomm when it comes to open source support for their phones. Qualcomm probably shouldn't have promised Linux support in the first place. Everyone seems to love Apple's hardware even though you're practically stuck with macOS. Had Qualcomm just stuck to Windows-only, they would've probably received a much better reception by the tech press.
- emsign 5mo agoThey are useless if RAM prices are this high. $800 laptops with maximum 8GB are currently the norm, Windows 11 can't run on them decently. No matter how fast the SoC is with overpriced RAM they are slow. Systems that can make good use of them with 64-128GB are not affordable anymore thanks to Nvidia and co. This is a smokescreen. They'll probably sell them packaged as compute modules anyway.
- alt227 5mo ago> Windows 11 can't run on them decently Windows 11 can run just fine on 8Gb of memory, what cant is Google Chrome.
- fg137 5mo agoI don't think this is going to get any traction in the general consumer world, even less relevant than Apple Vision Pro. (HN reaction to Vision Pro back in 2024 is almost hilarious if not ridiculous, looking at it today. I knew it would be a flop and I was so right.)
- monster_truck 5mo agoWild that it took me so much scrolling to find some sense! Spark DGX also remains a nothingburger, I would be livid if I spent this kind of money and had to waste time chasing down power cap bugs or A/B/C testing each firmware version to find the one that is least slow and also does not fail https://dredyson.com/the-hidden-truth-about-dgx-spark-performance-degradation-gpu-power-draw-issue-what-every-developer-needs-to-know-about-the-14w-power-cap-bug-proven-workarounds-and-the-complete-diagnostic-guide-t/ https://dredyson.com/the-hidden-truth-about-dgx-spark-perfor...
- PeterStuer 5mo ago"I am not sure how many people will run AI models locally. It still seems like a niche application to me." Clip me :). You are currently living through the final stages of unrestricted computing in the hands of the 'public'. Our regimes are going to pull up the drawbridge in the name of 'safety'. Download the open models asap and prepare for an airgapped computing environment. That will be your frontier in not extremely neutered AI in the near future. I am so hoping I'm completely wrong on this btw.
- shevy-java 5mo agoThey have bought the governments, so what you model will probably become true. They just inflate the prices right now.
- deleted 5mo ago[deleted]
- shevy-java 5mo agoAnd it will be expensive - right? Nvidia is milking the market now. We need more competition again - currently we have a mafia control the prices, not just Nvidia but all the AI companies. The price increases should be paid for them, not by us. "Free market" is being manipulated by them here.
- derefr 5mo ago> The game changer is the unified 128 GB memory. That is the path Apple took years ago. Instead of separate memory for the CPU and GPU, everything shares a single pool. It is increasingly popular. > The memory is not as fast as dedicated GPU memory, but it is cheap enough while delivering enough bandwidth to run AI models locally. So, the reason "dedicated GPU memory" is fast, isn't because it's "dedicated"; it's because the types of memory built into GPU cards — GDDR and HBM — are designed for throughput over latency. Which is to say, GDDR and HBM memory could be shared with the CPU in UMA while still being "fast" (for GPU use-cases.) In fact, the PS4/5 and Xbox 360 / One X / Series consoles have UMA architectures that use GDDR memory as their main memory, with no regular DDR memory to be found. What I don't understand: why don't we see UMA architectures where there's both regular DDR and GDDR/HBM memory mapped into the address space of the CPU+GPU? That seems like the best of both worlds: you'd have some memory that's "tuned" for random-access CPU usage (regular DDR), and some memory that's "tuned" for streaming GPU usage (GDDR/HBM), but either type of memory can still be put to the use it wasn't "tuned" for, just with slightly-worse performance. I guess you'd need to do a bit of software work: 1. a bit of work in the OS kernel / malloc library to get CPU workloads to "prefer" allocating DDR memory over the GDDR/HBM memory until they've exhausted DDR memory (or maybe not, if you just tell the kernel the GDDR/HBM memory is something like a zswap thinpool); 2. and a bit of work in supported ML frameworks, to teach them about a hybrid strategy between UMA "allocate anywhere, it's all the same" and NUMA "keep assets in VRAM if possible; if you spill assets to RAM, then they must stream into VRAM on access" (i.e. "at allocation time, allocate as if the system were NUMA, VRAM first then spilling to RAM; but at execution time, use the UMA codepaths, no need to copy RAM into VRAM.") ...but once that's done, it's done.
- Rohansi 5mo agoTheoretically, maybe? But they are completely different interfaces so it would surely get complicated. It's also approaching the current behavior in non-unified memory systems where you have two pools of memory with different performance characteristics. You'll realistically want the CPU to always use low latency memory and the GPU to use high bandwidth memory with very little moving between them.
- amacbride 5mo agoIt's effectively the same as the GB10 in the DGX Spark (Blackwell architecture, 6,144 CUDA cores, perf-wise comparable to an RTX 5070). I've found it very useful for running big models, but it's not a screaming powerhouse in terms of raw compute.
- adamnemecek 5mo agoThey are early versions, wait 4 years.
- stego-tech 5mo agoThe Unified Memory pool is what will continue to be the “game changer” in systems architecture, especially outside of data centers. The reality is even cutting edge games and consumer workloads don’t actually take full use of the PCIe bandwidth of the GPU or the bandwidth of its GDDR memory. Even local AI use cases don’t substantially or meaningfully benefit from faster memory, at least to average consumers. A unified memory pool does two things: 1) Lets systems optimize utilization based on need, rather than be confined to specific pools 2) Reduce overall memory cost, by letting system builders purchase a single type of memory in bulk instead of having to figure out GDDR vs DDR memory placement (important for SFF/portable machines) So at a time when memory is expensive, unified pools make more sense. Even when memory becomes cheap and plentiful again, it’s just practical at this point to allocate a larger overall pool instead of managing discrete sets. The one big drawback is security. A shared memory pool means side-channel attacks against memory from the GPU or CPU could potentially compromise the other as well, meaning memory-safe designs are going to be critical to security going forward (which is good for Rust adherents, I figure).
- Asmod4n 5mo agoyeah, you only see double digits in performance degradation from going from pcie 5 to 3 with a 5090 (at x16 speed), with everything else its like in the single digits area.
- stego-tech 5mo agoAnd the thing we gamers forget is that we’re the outlier. We’re the edge case. Most consumers will never really care about, let alone see, the difference in PCIe or memory bandwidth impacts from such a shift to unified memory pools. We might (being, at least in my case, a huge nerd), but I’m increasingly of the opinion that if modern blockbuster games are built for upscaling/reconstruction anyhow, then suddenly such sacrifices to performance seem acceptable relative to the gains in efficiency.
- jayd16 5mo agoWell I mean, the idea with games is it all fits in vram. You really don't want to be thrashing. It's that things are still so slow that they must be avoided entirely, no? No copy unified memory will help with that but you do pay the read speed costs.
- mariopt 5mo agoI think most people are not understanding what this kind of laptop will provide. Before we get local AI, we'll be using hybrid AI. Running big models locally is unrealistic ($$$$$) but, if you imagine an Agentic Workflow where some bits run on the cloud and other smaller tasks locally, it's an amazing deal. You don't need Opus/Code/DeepSeek/Kimi/etc to do basic stuff that models like Gemma4:12b/Qwen-27b can do locally with much less latency. Having a laptop where I can use a remote big model and combine it with 5 local domain specific models, is something I would love to do today. Imagine using OpenCode and you've a small model deciding which tasks run locally, then decides if you've a good local model for XYZ task or if we use a cloud model. My main concern is: Is this hardware powerfull enough to allow local quick models switch? Unlikely but I hope I'm wrong
- Gareth321 5mo agoGiven the incredible progress of local models, on present trajectory I think we see comparable levels of performance to frontier models in two years on 128GB unified RAM and 6-bit quantisation. Note how the frontier models are now hitting superior benchmarks with only 200,000 tokens. I think we still have a long way to go with distillation.
- zuzululu 5mo agoit should be but im more excited i can game on it
- ftchd 5mo agohttps://xcancel.com/lemire/status/2062880075117113739 https://xcancel.com/lemire/status/2062880075117113739
- proxysna 5mo agoIt's just a DGX spark with faster memory and a windows boot?
- epolanski 5mo agoNot gonna lie, I'm buying one of the 128GB ram ones for local inference if price is human.
- throwaway5752 5mo ago"Major banana producer suggests shifting more ice cream store menus to banana splits, and increasing the amount of bananas per serving"
- Animats 5mo agoHow much is this supposed to cost, fully populated with 128GB of RAM? How much would this laptop cost? It's not that the NVidia chip has that much RAM built in, after all. It's that it can address that much. RAM is sold separately.
- forrestthewoods 5mo agoNo. It’s all integrated. Not something you can buy separately or upgrade later.
- Animats 5mo agoSee [1]. There's not 128GB of on-chip memory. "Integrated" memory in this context means that the GPUs and CPUs all use the same memory. There are on-chip caches, of course. [1] https://www.nvidia.com/en-us/products/rtx-spark/ https://www.nvidia.com/en-us/products/rtx-spark/
- forrestthewoods 5mo agoEhhh. It is all in integrated into one monolith “superchip” package. The 128gb of RAM isn’t going to be purchased separately or be upgradable. At least according to all indicators. Which is what I was responding to.
- wmf 5mo ago$4,000-5,000
- cthalupa 5mo agoThe dgx spark is the same chip and those are in the low 3s to 5 range for most of them depending on manu, storage config, etc. The dgx sparks also have connectx 7 cards in them to support the 200gbps networking for RoCE. So I would expect the mini PCs to come in less than the sparks. Laptops I assume will be close in price with the addition of all the other laptop stuff.
- GodelNumbering 5mo agoI do not see how it is a 'beast of a' anything. It has 300GB/s memory bandwidth, barely above AMD Strix halo (256GB/s) with the same 128GB RAM and less than half memory bandwidth of M5 Max 128GB (614GB/s). Emphasizing memory bandwidth because most people interested in it I suppose are AI enthusiasts. Also, Windows.
- moffkalast 5mo agoThey have a lot of software groundwork ahead of them to make an ARM CPU viable for any kind of desktop use outside direct inference or training usage too. AMD has the advantage that their x86 machines run everything, Apple maintains the whole MacOS stack, while Nvidia can barely scrape together one Ubuntu release per Jetson generation, it's beyond embarrassing. Maybe they ought to put those agents they keep droning about to some actual work on their OS support.
- bigyabai 5mo ago> Nvidia can barely scrape together one Ubuntu release per Jetson generation Why would they do more? It's an LTS distro, the Nvidia drivers are updated for as long as the hardware's compute capability is supported. Nvidia's ARM drivers are updated constantly, and battle-tested as the backbone in hundreds of thousands of Grace ARM servers.
- moffkalast 5mo agoYes it's an LTS, which means it only makes sense to build something on it early in its release cycle, so the platform itself needs to keep up with new releases. Orins will go EoL next year, so good luck with that. That's not even considering the lazy out tree patchwork support Nvidia does for their products on top of that. Maybe it's different in this case for Windows since it forces a rolling release, but I seriously doubt they'll do it properly instead of forking some version and keeping it around for 10 years like absolute idiots.
- bigyabai 5mo ago
- xpct 5mo agoI follow Daniel Lemire and like his contributions, I also understand that the HN thread was created for discussion purposes, but I'd really appreciate having a reference to the spec or a source to the claims made, either here on HN or on the tweet itself. I dislike the cycle of propagating news and assuming that someone else double-checked it.
- Someone 5mo agoI think you’re asking for https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-pcs-agents-rtx-spark https://nvidianews.nvidia.com/news/nvidia-microsoft-windows-...: “News Summary: - NVIDIA RTX Spark powers the world’s first Windows PCs purpose-built for personal agents, featuring 1 petaflop of AI performance, industry-leading power efficiency, full-stack NVIDIA AI and graphics technology, and up to 128GB of unified memory. - NVIDIA and Microsoft collaborate to deliver a native Windows experience for personal agents, including new security primitives and NVIDIA OpenShell to run agents securely on primary devices. - RTX Spark lets creators, AI developers and gamers render ultralarge 90GB+ 3D scenes, edit 12K 4:2:2 video, generate 4K AI videos, run 120B-parameter LLMs with up to 1 million tokens context using agents locally, and play AAA games at 1440p and over 100 frames per second. - Adobe is rearchitecting Photoshop and Premiere from the ground up for RTX Spark to deliver 2x faster AI and graphics performance. - RTX Spark-powered slim Windows laptops with all-day battery life and premium displays, as well as compact desktop PCs available this fall from ASUS, Dell, HP, Lenovo, Microsoft Surface and MSI, with models from Acer and GIGABYTE to follow.”
- dh2022 5mo agoA beast if a Windows PC to do what? Run Trams, Excel, Outlook, and a browser all at the same time? We could do that just fine in 2010…
- mrweasel 5mo agoJust give it a year or two and Windows will drag that sucker into the mud and run everything just a sluggish as ever. The idea that any hardware performance increase will be eaten up by terrible software is an evergreen. A computer that could serve as the single server for a medium size enterprise 20 years ago, is no longer able to serve as a desktop for a receptionist. I'm not even sure we're talking diminishing returns anymore, we're probably past the point of maximum yield and into the negative returns at this point.
- kcb 5mo agoThe same chip that's been available in the DGX Spark for like 8 months now...why are we pretending like its the next big thing.
- danielovichdk 5mo agoA hardware company that propose to buy more hardware from them. Must be a new business model. .... Step into my office Why ? Because you are fucking fired
- dcreater 5mo agoThey announced RTX spark days ago. Why is this post linking to a "leak" tweet on the frontpage now?
- deleted 5mo ago[deleted]
- VortexLain 5mo agoI really hope this will have proper GNU/Linux support, otherwise it will end up the same way Qualcomm ARM PCs did.
- QQ00 5mo agoguaranteed it will be like Qualcomm arm, it's a partnership with Microsoft after all. we may see a community project to make Linux work on it but it will not have an official first class support and many things will likely not work properly.
- noveltyaccount 5mo ago> “Our goal is to deliver unmetered intelligence to every home and every desk with Windows,” said Satya Nadella, chairman and CEO of Microsoft. “RTX Spark marks a real breakthrough towards that vision.” I expect computers with this chip will be about $4000. If Microsoft can deliver on local AI models that can orchestrate Windows and have solid real world intelligence, that will be an inexpensive business purchase compared to pay as you go tokens. I'm excited to see how this plays out.
- daft_pink 5mo agoSounds good, but how much does it cost? Is this going to be an affordable laptop or $6000.
- shadowpho 5mo agoI mean fast 128gb of ram is like $2k so it’ll be $3k after overhead for just the ram portion of device.
- htk 5mo agoThe M1 Max from 2021 has better memory bandwidth. The M3 Max can be specced to 128GB. Nothing new here, apart from being able to use CUDA on a less power hungry system.
- bigyabai 5mo agoThe M1 Max has an unusably slow GPU for inference. TTFT on real-world contexts can be over 10 minutes. > Nothing new here, apart from being able to use CUDA on a less power hungry system. CUDA has been running on ARM SOCs since the Tegra K1, 12 years ago. Nvidia is not new to ARM, nor is CUDA.
- einpoklum 5mo agoIntel's basic architecture keeps accelerators away from main system memory, unlike, for example, IBM's POWER architecture where the CPU and GPU are equal 'users' of memory. It's not a great breakthrough to suggest something different. The problem is - it's different, and not compatible with a lot, or most, or all, existing hardware. Also, there are some security concerns, as @stego-tech noted.
- buffer_overlord 5mo agoCan it run Ubuntu?
- neuroelectron 5mo agoIt's going to be amazing. Almost twice as fast for only 10 times the heat. Consumers aren't concerned with efficiency they only care about performance.
- comandillos 5mo agoI dont really get the hype with all the N1X thing when in reality this is the same almost 1 yr old GB10 that was released with the DGX Spark and proved to be quite a disappointment
- siliconc0w 5mo agoI can't really see wide adoption of local LLMs unless prices really start to climb. It makes sense to use cheaper hosted smaller models like Sonnet or even Kimi but these won't run a Kimi-class model and that is really the floor for non-toy agentic tasks. Spending 5k to avoid a $20 subscription really only makes sense for niche security reasons.
- zzzoom 5mo agoI'd bet on the inverse: China scaling DRAM production until the price crashes, and the whole US stock market that is propped on top of that scarcity going down with it.
- maipen 5mo agoTarrifs.
- cthalupa 5mo agoPlenty of demand outside the US. Why would the hyperscalers not buy the chinese RAM for all of their datacenters across the world besides the US one? Rising supply from China will impact prices even in countries where there are tariffs.
- carefree-bob 5mo agoThe best Chinese RAM on the market is 50% larger and requires more power and thus emits more heat, as it is a 16nm feature size. If they can get to competitive sizes, then of course data centers will purchase it.
- mathgladiator 5mo agoI think we really haven't even seen how GenAI can influence new products and games. Have you consumed Dungeon Crawler Carl?
- 5mo ago
- vegabook 5mo agoI ran two gens of Jetson board and I have zero confidence in this. NVDA is printing in the data centre and everything else has no staying power.
- bigyabai 5mo agoI've ran two gens of Jetson, two gens of Nvidia/Linux dGPU and one gen of Tegra. The dGPUs have much better drivers than the devkit hardware does.
- mohamedkoubaa 5mo agoThe commenters here seem to have forgotten that computers can do things other than inference
- oldnetguy 5mo agoSGI had unified memory back in 1996.
- yoyohello13 5mo agoIf it runs well with Linux, I’m sold. A Windows pc will never see the inside of my network.
- seabrookmx 5mo agoThe DGX spark desktop shipped with Ubuntu, but I haven't seen if it has a bunch of Nvidia specific repos for drivers etc. needed to make it function. Assuming all that stuff is upstreamed (and they aren't using oddball webcam/input devices etc) it should have much better support than Qualcomm. Fingers crossed!
- Schnitz 5mo agoHow is this different from something like the AMD Ryzen AI Max that can already be purchased and supports 128GB unified memory? Seriously curious.
- nine_k 5mo agoMaybe CUDA support, or something else specific to NVidia?
- JBiserkov 5mo agoOfftopic, but "Twitter. Now that's a name I've not heard in a long time. A long time".
- snvzz 5mo agoIt is not RISC-V. We aren't so naive as to move from a locked IP ISA like x86 to another locked IP ISA such as ARM. Right?
- gghh 5mo agoSorry for lazy question but some people here know this off top of their head so asking. Memory bandwidth of this chip? Last time I check an NVidia situation was for DGX Spark (the GB10 chip), it has regular LPDDR5X which by JEDEC standard cannot go beyond ~270 GB/sec, ie 8533 Mbit/s on a 256 lanes bus. So yeah Lemire seems to go "OMG unified memory, they're following Apple path..." ok, but Apple pulled off a much faster interconnect, 800 GB/s ballpark, and I'm trying to understand (not really, I'm asking you to try understand, he he) how is this laptop faring in that regard.
- sylware 5mo ago[dead]
- 1970-01-01 5mo agoLocal models becoming thousands of dollars instead of millions to run is a story the public genuinely seems to be unaware of. If the order of magnitude falls again, the markets are cooked. The cheap chips barrier is even artificial and unsustainable. The next big story in local AI adoption will be big players doing chip hoarding-as-a-strategy.
- wombat-man 5mo agoI think this is part of why capex is so high for the big players. Yes data centers are expensive to build but it's also about outbidding for the labor and materials needed to build those. They'd rather build compute they don't need and then lease it then let competitors have their own machines.
- iceflinger 5mo agoI'm fully convinced now that this is the real reason for the OpenAI RAM wafer deals.
- effnorwood 5mo agobeast is right
- Melatonic 5mo agoWe need a physical architectural change if we really want to improve this. You can only physically wire so much memory next to a chip in a 2 dimensional (flat) design - limited by the edge lengths of the chip (CPU or GPU) Basically what we need is a chip that also has pins or some type of attachement system on the top (physically) or maybe below where the chip itself connects to the motherboard. Imagine a CPU you can just plug in a block of HBM memory on top of (or on "bottom" of). This would allow a much larger physical surface area for putting ram cache near the compute cores itself because you would not be limited to edge lengths. Cooling the whole thing would be a methodology change (might need liquid coolers that sandwhich in between the ram cache and compute and cool both)
- Melatonic 5mo agoWhat does he mean at the end when he talks about Intel processors - is he referencing something supported in AVX-512 that is unique to AMD or ?
- empiree 5mo agoInteresting move. More competition in CPUs is good for everyone. My only question is software support, that's usually where these non-x86 launches stall. Hoping it's more than a spec-sheet flex.
- mindcrash 5mo ago> "The game changer is the unified 128 GB memory. That is the path Apple took years ago. Instead of separate memory for the CPU and GPU" Not really a game changer as AMD already has this (but is quite terrible at marketing it). This is the reason why you can run pretty big AI models on reasonable speeds on embedded GPUs on the Ryzen AI SoCs and get even better performance on the Ryzen AI Max+ SoCs. As a example I have a HX370 based laptop with 128G RAM and due to these specs can instruct the included firmware to share up to 96G (or approximately 66% of available system memory) with the embedded GPU, which gives quite some headroom to experiment with local models as said before. The included GPU with 6144 CUDA cores isn't really big news either to be honest, as the DGX Spark already has them too. Likely because they simply integrated the same (or very similar) GPU on this SoC. In fact the whole thing looks eerily similar to the DGX Spark SoC, so the only new thing pretty much seems to be the support for Windows.