15 ms·
Nvidia DGX Spark
- oracel 1y agoCan it run Crysis?
- brookst 1y agoIt can write Crysis.
- mynegation 1y agoIt’s gonna run Crysis for my wallet alright
- bigyabai 1y agoBit of a non-sequitur, but it seems like it's been playable via box86 for quite a while now: https://youtu.be/NcBJG3z8kF0 https://youtu.be/NcBJG3z8kF0
- TiredOfLife 1y agoIf raspberry pi can then this probably also can. https://www.jeffgeerling.com/blog/2024/amd-radeon-pro-w7700-running-on-raspberry-pi https://www.jeffgeerling.com/blog/2024/amd-radeon-pro-w7700-...
- nightski 1y agoAm I missing something or does the comparably priced (technically cheaper) Jetson Thor have double the PFLOPs of the Spark with the same memory capacity and similar bandwidth?
- modeless 1y agoAlso Thor is actually getting sent out to robotics companies already. Did anyone outside Nvidia get a DGX Spark yet?
- Apes 1y agoMy understanding is the DGX Spark is optimized for training / fine tuning and the Jetson Thor is optimized for running inference. Architecturally, the DGX Spark has a far better cache setup to feed the GPU, and offers NVLINK support.
- AlotOfReading 1y agoThere's a lot of segmentation going on in the Blackwell generation from what I'm told.
- deleted 1y ago[deleted]
- lvl155 1y agoIs this worth getting vs AMD?
- zxexz 1y agoWhat are you trying to do?
- manishsharan 1y agoNot the above poster. I would like to run agents with local multimodal LLMs to process huge numbers of sensitive files for an org( summarization, knowledge extraction, answer user questions ,etc). Any ideas?
- DoctorOetker 1y agosuppose 1/3rd of memory is used to host a teacher network, and 2/3rds of memory is used to host a student network, how long would knowledge distillation typically take?
- blitzar 1y agoThey will meet at 18:47 on tuesday evening.
- Y_Y 1y agoIt's a bit disingenuous to claim 1 PFLOPs without making clear that's for FP4 (with "structured sparsity"?)
- csunoser 1y agoIt does say `Experience up to 1 petaFLOP of AI performance at FP4 precision with the NVIDIA Grace Blackwell architecture.` in the features section. But yeah, this should have been further up.
- godelski 1y agoIf you scroll down a little and see the chip icon, where it says "NVIDIA GB10 Superchip " it also says "Experience up to 1 petaFLOP of AI performance at FP4 precision with the NVIDIA Grace Blackwell architecture." Further down, in the exploded view it says "Blackwell GPU 1PetaFLOP FP4 AI Compute" Then further down in the spec chart they get less specific again with "Tensor Performance^1 1 PFLOP" and "^1" says "1 Theoretical FP4 TOPS using the sparsity feature." Also, if you click "Reserve Now" the second line below that redundant "Reserve Now" button says "1 PFLOPS of FP4 AI performance" I mean I'll give you that they could be more clear and that it's not cool to just hype up on FP4 performance, but they aren't exactly hiding the context like they did during GTC. I wouldn't call this "disingenuous"
- Y_Y 1y agoEven if that "sparsity feature" is that two or of every four adjacent values in your areay be zeros, and that performance halves if not doing this? I think lots of children are going to be very disappointed running their blas benchmarks on Christmas morning and seeing barely tens of teraflops. (For reference see how the still optimistic numbers are for the H200 when you use realistic datatypes. https://nvdam.widen.net/s/nb5zzzsjdf/hpc-datasheet-sc23-h200-datasheet-3002446 https://nvdam.widen.net/s/nb5zzzsjdf/hpc-datasheet-sc23-h200... )
- imtringued 1y agoUsing sparsity in advertising is incredibly misleading to the point of lying. The entire point of sparsity is that you avoid doing calculations. Sparsity support means you need fewer FLOPs for a matrix of the same size. It doesn't magically increase the number of FLOPs you have. Even AMD got that memo and is mostly advertising their 8bit/block fp16 performance on their GPUs and NPUs, even though the NPUs support 4 bit INT with sparsity, which would 4x the quoted numbers if they used Nvidia's marketing FLOPs.
- MBCook 1y agoI’m not in this space, so I don’t know what’s normal, but I guess I’m a little surprised to see only 10 gig Ethernet for high speed connectivity. Yeah, it’s miles better than WiFi. But if there was something I’d think maybe benefit from Thunderbolt this would’ve been it. The ability to transfer large models or datasets that way just seems like it would be much faster and a real win for some customers.
- coder543 1y agoThis thing has a ConnectX-7, which gives it 2 x 200 Gbps networking. The 10 gig port is far from the fastest network interface on the Spark.
- MBCook 1y agoBut can you hook that up to a normal PC?
- coder543 1y agoYou were complaining about speed. Yes, a PC can have the same ports, and then you get much faster speeds than Thunderbolt can provide. Why would you ever want a DGX Spark to talk to a “normal PC” at 40+ Gbps speeds anyways? The normal PC has nothing that interesting to share with it. But, yes, the DGX Spark does have four USB4 ports which support 40Gbps each, the same as Thunderbolt 4. I still don’t see any use case for connecting one of those to a normal PC.
- renewiltord 1y agoYes. Just buy the Mellanox card. We had a bunch of ConnectX 5 hooked up through SFP. Needs cooling but fast.
- _zoltan_ 1y agowhy criticize something in the first place when you clearly have not even looked at the product?
- deleted 1y ago[deleted]
- ls612 1y agoIs this the $3500 one?
- x2tyfi 1y agoThat was their Digits box.
- jrgifford 1y agoDigits is no more, it’s DGX. Source: signup link in the press release (https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...) now goes to DGX Spark preorder (https://www.nvidia.com/en-us/products/workstations/dgx-spark/ https://www.nvidia.com/en-us/products/workstations/dgx-spark...)
- wmf 1y agoIt's the same thing. They renamed it from Digits to Spark.
- gardnr 1y agoThere are cheaper ASUS and MSI versions "coming soon" with the same chip and less storage / memory.
- canucker2016 1y agoon the reserve now page for USA, there's: ASUS Ascent GX10 - 1TB $2,999 MSI EdgeXpert MS-C931 - 4TB $3,999 the 1TB/4TB seems to be the size of the included NVMe SSD. the reserve now page also lists NVIDIA DGX Spark Bundle 2 NVIDIA DGX Spark Units - 4TB with Connecting Cable $8,049 The DGX Spark specs lists an NVIDIA ConnectX-7 Smart NIC which is rated at 200Gbe to connect to another DGX Spark, for about double the amount of memory for models.
- dirtyhand 1y agoI was considering getting an RTX 5090 to run inference on some LLM models, but now I’m wondering if it’s worth paying an extra $2K for this option instead
- BoorishBears 1y agoNo. These are practically useless for AI. Their prompt processing speeds are absolutely abysmal: if you're trying to tinker from time to time, a GPU like a 5090 or renting GPUs is a much better option. If you're just trying to prep for impending mainstream AI applications, few will be targeting this form factor: it's both too strong compared to mainstream hardware, and way too weak compared to dedicated AI-focused accelerators. - I'll admit I'm taking a less nuanced take than some would prefer, but I'm also trying to be direct: this is not ever going to be a better option than a 5090.
- aurareturn 1y agoTheir prompt processing speeds are absolutely abysmal They are not. This is Blackwell with Tensor cores. Bandwidth is the problem here.
- BoorishBears 1y agoThey're abysmal compared to anything dedicated at any reasonable batch size because of both bandwidth and compute, not sure why you're wording this like it disagrees with what I said. I've run inference workloads on a GH200 which is an entire H100 attached to an ARM processor and the moment offloading is involved speeds tank to Mac Mini-like speeds, which is similarly mostly a toy when it comes to AI.
- aurareturn 1y agoAgain, prompt processing isn't the major problem here. It's bandwidth. 256GB/s bandwidth (maybe ~210 in real world) limits the tokens per second well before prompt processing. Not entirely sure how your ARM statement matters here. This is unified memory.
- senectus1 1y agoPower consumption : TBD ?? this seems more than a little disingenuous...
- canucker2016 1y agofrom https://www.servethehome.com/this-is-the-asus-ascent-gx10-a-nvidia-gb10-mini-pc-with-128gb-of-memory-and-200gbe/ https://www.servethehome.com/this-is-the-asus-ascent-gx10-a-... ASUS and NVIDIA told us that their GB10 platforms are expected to use up to 170W. [edit] the PSU is 240W so that'd place an upper limit on power draw, unless they upgrade it.
- wewewedxfgdf 1y agoIt'll be stunted in some way - Nvidia always holds back some crucial feature that you need, to push you up to the next highest priced product line.
- agnokapathetic 1y agoit uses LPDDR5x instead of the datacenter variant’s HBM3e.
- syntaxing 1y agoWhile a completely different price point, I have a Jetson Orin Nano. Some people forget the kernels are more or less set in stone for product like these. I could rebuild my own Jetpack kernel but it’s not that straight forward to update something like CUDA or any other module. Unless you’re a business where your product relies on this hardware, I find it hard to buy this for consumer applications.
- coredog64 1y agoCame in here to say the same thing. Have bought 3 Nvidia dev boards and never again as you quickly get left behind. You're then stuck compiling everything from scratch.
- larodi 1y agoMy experience with Jetson Nano was that it had to have its Ubuntu debloatred first (with 3rd party script) before we could get their NN something library to run the image recognition, designated to run on this device. These seem to be highly experimental boards, even though are super powerful for their form factor.
- syntaxing 1y agoThat’s true for the Jetson Nano. The Jetson Orin Nano (I know, the naming sucks) is much better in that aspect. Higher memory (8GB), way higher memory bandwidth (120 GB/s), and Orin has way more CUDA cores. It can pretty much run any “traditional” neural network, even YOLO large and even LLMs.
- cherioo 1y agoThe mainstream options seem to be Ryzen AI Max 395+, ~120 tops (fp8?), 128GB RAM, $1999 Nvidia DGX Spark, ~1000 tops fp4, 128GB RAM, $3999 Mac Studio max spec, ~120 tflops (fp16?), 512GB RAM, 3x bandwidth, $9499 DGX Spark appears to potentially offer the most token per second, but less useful/value as everyday pc.
- aurareturn 1y agoMac Studio max spec, ~120 tflops (fp16?), 384GB RAM, 3x bandwidth, $9499 512GB. DGX has 256GB/s bandwidth so it wouldn't offer the most tokens/s.
- rz2k 1y agoPerhaps they are referring to default GPU allocation that is 75% of the unified memory, but it is trivial to increase it.
- jauntywundrkind 1y agoThe GPU memory allocation refers to how capacity is alloted, not bandwidth. Sounds like the same 256-bit/quad-channel 8000MHz lpddr5 you can get today with Strix Halo.
- rz2k 1y ago384GB is 75% of 512GB. The M3 Ultra bandwidth is over 800GB/s, though potentially less in practice. Using an M3 Ultra I think the performance is pretty remarkable for inference and concerns about prompt processing being slow in particular are greatly exaggerated. Maybe the advantage of the DGX Spark will be for training or fine tuning.
- vid 1y agoI very consistently see people say prompt processing is slow for larger context sizes ("notoriously slow"), something that is much less of an issue with eg CUDA setups.
- ComplexSystems 1y agoThe RAM bandwidth is so slow on this that you can barely train or do inference or do anything on it. I think the only use case they have in mind for this is fine tuning pretrained models.
- wmf 1y agoIt's the same as Strix Halo and M4 Max that people are going gaga about, so either everyone is wrong or it's fine.
- gardnr 1y agoMemory Bandwidth: Nvidia DGX: 273 GB/s M4 Max: (up to) 546 GB/s M3 Ultra: 819 GB/s RTX 5090: ~1.8 TB/s RTX PRO 6000 Blackwell: ~1.8 TB/s
- 7thpower 1y agoThe other ones are not framed as an “AI Supercomputer on your desk”, but instead are framed as powerful computers that can also handle AI workloads.
- aurareturn 1y agoM4 max has more than double the bandwidth. Strix Halo has the same and I agree it’s overrated.
- Rohansi 1y agoI would expect/hope that DGX would be able to make better use of its bandwidth than the M4 Max. Will need to wait and see benchmarks.
- aurareturn 1y agoIt should. It has tensor cores which should drastically improve prompt processing. It should also be highly optimized for most AI apps.
- 1y ago
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- hereme888 1y agoFP4-sparse (TFLOPS) | Price | $/TF4s 5090: 3352 | 1999 | 0.60 Thor: 2070 | 3499 | 1.69 Spark: 1000 | 3999 | 4.00 ____________ FP8-dense (TFLOPS) | Price | $/TF8d (4090s have no FP4) 4090 : 661 | 1599 | 2.42 4090 Laptop: 343 | vary | - ____________ Geekbench 6 (compute score) | Price | $/100k 4090: 317800 | 1599 | 503 5090: 387800 | 1999 | 516 M4 Max: 180700 | 1999 | 1106 M3 Ultra: 259700 | 3999 | 1540 ____________ Apple NPU TOPS (not GPU-comparable) M4 Max: 38 M3 Ultra: 36
- aurareturn 1y agoIt's not good value when you put it like that. It doesn't have a lot of compute and bandwidth. What it has is the ability to run DGX software for CUDA devs I guess. Not a great inference machine either.
- jacquesm 1y agoIt's great at one thing: memory. And that's interesting because memory is a commodity, but they still make bank on just being able to access it.
- aurareturn 1y agoMemory is a commodity but access to high bandwidth memory is expensive whether it's HBM, or LPDDR/DDR connected to many memory channels.
- throwaway48476 1y agoIts not priced in a linear way wrt to bom cost.
- aurareturn 1y agoThat's because memory channels cost money. Memory controllers are more complex. Lastly, chips that can make use of high bandwidth VRAM are both of the above. Memory chips are a commodity, that I agree. Though HBM is trending towards not being a commodity.
- garyfirestorm 1y agoWhat did I miss? This was revealed in May - I don’t see anything new in that link since it was revealed.
- wmf 1y agoNot much. There was a presentation yesterday but it's mostly what we already knew: https://www.servethehome.com/nvidia-outlines-gb10-soc-architecture-at-hot-chips-2025/ https://www.servethehome.com/nvidia-outlines-gb10-soc-archit...
- numpad0 1y agoThis has been getting delayed for months. Display out isn't working or something.
- eadwu 1y agoMost people are missing the point. LLMs are not the be all end all of AI. Even if you were to say memory bandwidth was the problem, there is no consumer grade GPU that can run any SoTA LLM, no matter what you'd have to settle for a more mediocre model. Outside of LLMs, 256 GB/s is not as much of an issue and many people have dealt with less bandwidth for real world use cases.
- gardnr 1y agoWhat other use cases would use 128GB VRAM but not require higher throughput to run at acceptable speeds?
- Auracle 1y agoFine tuning text to image/video models perhaps? For the newest models unless you quantize the crap out of them, even with a 5090 you’re going to be swapping blocks, which slows things down anyways. At least you’d be able to train on them at full precision with a decent batch size. That said, I can’t imagine there’s enough of a market there to make it worth it.
- eadwu 1y agoPeople have done more with less for a long time (basically with the Jetson counterparts). The only likely difference with DGX Spark is that it'll be a more desktop-centered platform, what people can do with it, not sure, but say for VR, the DGX Spark is basically the best compute puck for one right now.
- sorrythanks 1y agoNVIDIA DGX Spark - 4TB $3,999
- numpad0 1y agoDo anyone know why official pages don't mention FP16 performance(250 TFLOPS)?
- mugivarra69 1y ago[dead]
- maz1b 1y agoDunno, doesn't seem that good to me. Granted, I recognize the pace of advancement, but fwiw at present time.. yeah. I'd rather just get an M3 Ultra. Have an M2 Ultra on the desk, and an M3 Ultra sitting on the desk waiting to be opened. Might need to sell it and shell out the cash for the max ram option. Pricey, but seems worthwhile.
- DrNosferatu 1y agoNow we need a threeway benchmark between this DGX Spark, a maxed out AMD Strix* and the Mac 512GB.
- maddynator 1y agoSo Raspberry Pi With GPU?
- _zoltan_ 1y agothe GH/GB line is also based around Arm. the CPU here doesn't matter.
- KingOfCoders 1y agoI think it depends on your model size Fits into 32gb: 5090 Fits into 64gb - 96gb: Mac Studio Fits into 128gb: for now 395+ $/token/s, Mac Studio if you don't care about $ but don't have unlimited money for Hxxx This could be great for models that fit 128gb and you want best $/token/s (if it is faster than a 395+).
- timc3 1y agoThe 395 although it can be supplied with 128GB can’t use all that for VRAM (unless something has changed in the last couple of weeks).
- lhl 1y agoIn Linux, you can set it as high as you want, although you should probably have a swap drive and still be prepared for you system to die if you set it to 128GiB. Here's how you'd set it to 120GiB: # This is deprecated, but can still be referenced options amdgpu gttsize=122800 # This specifies GTT by # of 4KB pages: # 31457280 * 4KB / 1024 / 1024 = 120 GiB options ttm pages_limit=31457280
- KingOfCoders 1y agoFrom YouTube it seems up to 105gb Models disksize work, yes.
- monster_truck 1y agoPaper launch. The people I know there who I have asked about it haven't even seen one yet
- fh973 1y agoOrdered one in spring. Delivery time was pushed from July to September. Apparently they had a bug in the HDMI output.
- wtallis 1y agoThat's eerily similar to what happened to Qualcomm's failed Snapdragon X Elite dev kit. That one eventually shipped in small quantities with a Type-C to HDMI dongle in the box to make up for the built-in HDMI port going missing. Then Qualcomm cancelled the whole project and refunded everyone, including people who had already received their hardware.
- _zoltan_ 1y agobecause they realized it sucked.
- throwaway48476 1y agoBecause the dev units shipped after retail units. Total incompetence from QC just as I predicted.
- isusmelj 1y agoAre there any news about power consumption? I didn’t even see a tdp or so mentioned.
- lmeyerov 1y agoOne of the first things I looked at too...
- canucker2016 1y agofrom my comment elsewhere in this thread, https://news.ycombinator.com/item?id=45048078 https://news.ycombinator.com/item?id=45048078, "up to 170W." was the quote from March.
- querez 1y ago"developers can prototype, fine-tune, and inference [AI models]"... shouldn't it be infer?
- nharada 1y agoI don’t think either of those are right…
- globular-toast 1y agoPerhaps "infer from"? I was also taken aback by how they just decided to make "inference" a verb, though. A decent writer would have rewritten the sentence to make it work, similar to how a software implementation sometimes just doesn't work out. But apparently that's too much to ask from Nvidia marketing. Funnily enough things like this show that a human probably was involved in the writing. I doubt an LLM would have produce that. I've often thought about how future generations are going to signal that they are human and maybe the way will be human language changing much more rapidly than it has done, maybe even mid sentence.
- myrmidon 1y agoIt should be "run inference on" in my opinion, and would be best shortened IMO to just "prototype, fine-tune, and run". I'argue that "inference" has taken on a somewhat distinct new meaning in an LLM-context (loosely: running actual tokens through the model) and deviating from the base term to the verb form would make the sentence less clear to me.
- killerstorm 1y agoNo. It's quite common for technical slang to deviate from general vocabulary. Cf. "compute" is a verb for normal people, but for techies it is also "hardware resources used to compute things".
- querez 1y agoI don't think "inference" as a verb has become technical slang. At least not in my bubble.
- apples_oranges 1y agoQuestion from a random consumer: Why not more RAM?
- seanalltogether 1y agoDo we need a new term to describe "unified memory" where the cpu and gpu are still isolated from each other and memory needs to be allocated for one or the other and "unified memory" where the cpu and gpu can both access the same addresses. Which systems use which?
- qwertox 1y agoAccording to Wendell from Level1Techs, the now-launched Jetson Thor uses a Linux Kernel built by Nvidia, on Ubuntu 20.04 [0]. So I assume getting upgrades will have the same feel as those Chinese SBC's like from Radxa or cheap Android devices. I wonder if this also applies to this DGX Spark. I hope not. [0] https://www.youtube.com/watch?v=cgnKUUcCKcs&t=669s https://www.youtube.com/watch?v=cgnKUUcCKcs&t=669s
- rowanG077 1y agoOooof that would be an instant dealbreaker for me... Better get a mac pro with asahi linux. That at least has great linux support.
- fh973 1y agoMarketing material says NVIDIA DGX™ OS, which at version 7 would be an Ubuntu 24.04: https://docs.nvidia.com/dgx/dgx-os-7-user-guide/introduction.html https://docs.nvidia.com/dgx/dgx-os-7-user-guide/introduction...
- naikrovek 1y agoor how the Jetson Nano was on Ubuntu 18.04 when it was released in 2019 and never got a single major OS upgrade. That platform was great for a few months...
- bri3d 1y agoIn the case of Jetson, NVidia also have a fairly generic BSP which you can use to customize almost any distribution, and the Jetson-customized Ubuntu is standard enough that you can upgrade it using the normal Ubuntu upgrade path without major issue. For most of the Tegra boards there’s also upstream support. Overall the situation with NVidia BSP is about 10000x better than weird Chinese stuff. In the case of Tegra/Jetson, there’s even detailed first-party documentation about reconstructing the BSP components from source: https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3275/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide/kernel_custom.html%23 https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-327... I’d assume the decent software support will carry over to DGX.
- ionwake 1y agoIm confused the landing pade to reserve the DGX spark was open 4 months ago? Or am I missing something there.
- DebtDeflation 1y agoIt's a great idea that they utterly gimped with the 273 GB/s memory bandwidth. You can buy a Macbook Pro M4 Max with 128GB that has TWICE the bandwidth. A 4090 has 4 TIMES the bandwidth. I get that they're terrified of doing anything that competes with their H100/B200 but there is an enormous gap between 273 GB/s and the >2TB/s of their data center products.
- throwaway48476 1y agoIts absurd because the B200 are all on NVLink. Different market entirely.
- penguin202 1y ago[dead]
- hoppp 1y agoSo what is the price?
- throwaway48476 1y agoThe only useful part of this is the new LPDDR memory form factor.
- pjs_ 1y agoSo if I buy 1000 of these I have an exascale supercomputer? I remember when exascale was disparaged as science fiction :)
- deleted 1y ago[deleted]