5 ms·
Cheaper? The DGX spark is $3,000 with 128GB ram. A framework desktop with the AMD strix halo 395 with 128gb ram is $2,000 and has better memory bandwidth. No
by sliken 2y ago
Cheaper? The DGX spark is $3,000 with 128GB ram. A framework desktop with the AMD strix halo 395 with 128gb ram is $2,000 and has better memory bandwidth. No price I can see for the identically spec'd (and nearly identical looking) Ascent gx10.
[edit] Oops, the Spark is $4,000, only the Ascent is at $3k now. Strix Halo systems vary from slightly slower (6%) to the same (on systems with LPDDR5x-8533, like the HP laptop).
- walterbell 2y agoIncludes RDMA 200GbE NIC. > NVIDIA ConnectX-7 NIC these days often sells for $1500-2200 in single unit quantities, depending on the features and supply of the parts. At $2999 for a system with this buit-in that is awesome.
- kcb 2y agoYea unlike other options this is actually scalable. The question isn't if 1 can outperform the Mac Studio but if 3-4 linked together can.
- Dylan16807 2y agoWhat kind of scaling do you have in mind there? My naive analysis is: A high end Mac should be able to run each layer of an AI task about twice as fast because of the memory bandwidth. And the data going between layers is tiny enough to run over thunderbolt or even normal ethernet. Is there an AI use case that prefers 250GB/s memory bandwidth plus 25GB/s interconnect over 500GB/s memory and 2GB/s interconnect? Are there other major use cases that prefer it?
- fancyfredbot 2y agoUsually the reason you'd want the network bandwidth would be for distributed training. For inference you can probably get by with 2GB/s assuming you can split the layers up nicely. The interconnect can be a bottleneck for inference but only for networks with loads of activations and large batch sizes, or if you are doing tensor level parallelism.
- sliken 2y agoSort of. They come in different speeds, and those prices are for the 400gbe version. The 200gbe, like in the GX10 and spark are more like $1250. Not to mention you have to need that the 200gbe (to cluster 2 of them) and I'd expect the vast majority to buy a single unit, not a pair.
- walterbell 2y agoWith a crossover cable, a single unit could be used for local testing of software that depends on both RDMA and CUDA.
- rubatuga 2y agoAgreed not cheaper by a long shot.
- wtallis 2y agoDouble check your memory bandwidth numbers. AMD's Strix Halo is 256 bits at 8000MT/s for about 238GB/s while NVIDIA's GB10 is 276GB/s (likely 256 bits at the more standard speed of 8533MT/s).
- sliken 2y agoDepends which one, the HP ZBook Ultra uses LPDDR5x-8533, a dead match for what nvidia claims (273GB/sec). Although the DGX spark now costs $4,000 instead of the "starting at $2,999" mentioned a few months ago. So the bandwidth is dead even between AMD and Spark.