6 ms·
Article doesn't seem to mention price which is $4,000 which makes it comparable to a 5090 but with 128GB of unified LPDDR5x vs the 5090's 32GB DDR7.
by SethTro 1y ago
Article doesn't seem to mention price which is $4,000 which makes it comparable to a 5090 but with 128GB of unified LPDDR5x vs the 5090's 32GB DDR7.
- CamperBob2 1y agoAnd about 1/4 the memory bandwidth, which is what matters for inference.
- threeducks 1y agoMore precisely, the RTX 5090 has a memory bandwidth of 1792 GB/s, while the DGX Spark only has 273 GB/s, which is about 1/6.5. For inference, the DGX Spark does not look like a good choice, as there are cheaper alternatives with better performance.
- CamperBob2 1y agoMy understanding is that the Jetson Thor is just as good a platform, and likely more readily available. Then there's the Mac Studio, which outdoes them in all respects except FP8 and FP4 support. As someone on Reddit put it: https://old.reddit.com/r/LocalLLaMA/comments/1n0xoji/why_cant_we_just_use_nvidia_jetson_agx_thor/nawxbk2/ https://old.reddit.com/r/LocalLLaMA/comments/1n0xoji/why_can...
- altspace 1y agoI’ve been thinking the same… I have jetson Thor and only difference I can imagine is the capability to connect two DGX sparks together… but then I’d rather go for RTX pro 6000 instead of buying two DGX spark units, because I prefer the higher memory bandwidth, more Cuda cores, tensor cores and RT cores over 256 GB memory for my use case.
- KeplerBoy 1y agoThe jetson thor seems to be quite different. The Thor whitepaper lists 8 TFlop/s of FP32 compute where the DGX sparks seems to be closer to 30 TFlop/s. Also 48 SMs on the Spark vs 20 on the Jetson. The DGX seems vastly more capable.
- nialse 1y agoWell, that’s disappointing since the Mac Studio 128GB is $3,499. If Apple happens to launch a Mac Mini with 128GB RAM it would eat Nvidia Sparks’ lunch every day.
- moondev 1y agoJust don't try to run a NCCL
- zackangelo 1y agoWouldn't you be able to test nccl if you had 2 of these?
- newman314 1y agoAgreed. I also wonder why they chose to test against a Mac Studio with only 64GB instead of 128GB.
- yvbbrjdr 1y agoHi, author here. I crowd-sourced the devices for benchmarking from my friends. It just happened that one of my friend has this device.
- ggerganov 1y agoFYI you should have used llama.cpp to do the benchmarks. It performs almost 20x faster than ollama for the gpt-oss-120b model. Here are some samples results on my spark: ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GB10, compute capability 12.1, VMM: yes | model | size | params | backend | ngl | n_ubatch | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | -------: | -: | --------------: | -------------------: | | gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | CUDA | 99 | 2048 | 1 | pp4096 | 3564.31 ± 9.91 | | gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | CUDA | 99 | 2048 | 1 | tg32 | 53.93 ± 1.71 | | gpt-oss 120B MXFP4 MoE | 59.02 GiB | 116.83 B | CUDA | 99 | 2048 | 1 | pp4096 | 1792.32 ± 34.74 | | gpt-oss 120B MXFP4 MoE | 59.02 GiB | 116.83 B | CUDA | 99 | 2048 | 1 | tg32 | 38.54 ± 3.10 |
- EnPissant 1y agoA 5090 is $2000.
- sandworm101 1y agoMsrp, but try getting your hands on one without a bulk order and/or camping out in a tent all weekend. I have seen people in my area buying pre-biult machines as they often cost less than trying to buy an individual card.
- EnPissant 1y agoIt’s not that hard to come across MSRP 5090s these days. It took me about a week before I found one. But if you don’t want to put any effort or waiting into it, you can buy one of the overpriced OC models right now for $2500.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- adrian_b 1y agoBut you put in a $1500 PC (with 128 GB DRAM). Still, a PC with a 5090 will give in many cases a much better bang for the buck, except when limited by the slower speed of the main memory. The greater bandwidth available when accessing the entire 128 GB memory is the only advantage of NVIDIA DGX, while a cheaper PC with discrete GPU has a faster GPU, a faster CPU and a faster local GPU memory.
- bilekas 1y ago$4,000 is actually extremely competitive. Even for an at-home enthusiast setup this price is not our of reach. I was expecting something far higher, that said, nVidia's MSRP is something of a pipe dream recently so we'll see when it's actually released and the availability. Curious also to see how they may scale together.
- Xss3 1y agoA warning to any home consumer throwing money at hardware for AI (fair enough if you have other use cases)... Things are changing rapidly and there is a non insignificant chance that it'll seem like a big waste of money within 12 months.
- eadwu 1y agoFor this form factor it will be likely ~2 years for the next one based on Vera CPU and whatever GPU. The 50W CPU will probably improve power efficiency. If SOCAMM2 is used it will still probably be at most near the range of 512/768 GB/s bandwidth, unless LPDDR6X / LPDDR7X or SOCAMM2 is that much better, SOCAMM on the DGX Station is just 384 GB/s w/ LPDDR5X. Form factor will be neutered for the near future, but will probably retain the highest compute for the form factor. The only way there will be a difference is if Intel or AMD pump their foot on the gas, which this makes maybe 2/3 years of it, with another 2 years unless they have something cooking it isn't going to happen.
- Xss3 1y agoSoftware driven changes could occur too! Maybe the next model will beat the pants off of this with far inferior hardware. Or maybe itll be so amazing with higher bandwidth hardware that anyone running at less than 500gbs will be left feeling foolish. Maybe a company is working on something totally different in secret that we cant even imagine. The amount of £ thrown into this space at the moment is enormous.
- metadat 1y agoBased on what data? I'm not denying the possibility but this seems like baseless FUD. We haven't even seen what folks have done with this hardware yet.
- Tepix 1y agoThey're in a different ballback in memory bandwidth. The right comparison is the Ryzen AI Max 395 with 128GB DDR5-8000 which can be bought for around $1800 / 1750€.