3 ms·
So I can spend thousands of dollars to have an unstable training environment and inference performance worse than a US$200 3060. Wow. Where do I sign up?
by suprjami 1y ago
So I can spend thousands of dollars to have an unstable training environment and inference performance worse than a US$200 3060.
Wow. Where do I sign up?
- thehamkercat 1y agoThe memory bandwidth on this thing is absolute trash, better buy a mac mini/studio with this much ram if you're throwing this much money, it'll be faster (M4 Max)
- suprjami 1y agoAgree, any Max or Ultra should walk all over this thing, and has the advantage of many years of already-working software. Apple benchmarks: https://github.com/ggml-org/llama.cpp/discussions/4167 https://github.com/ggml-org/llama.cpp/discussions/4167
- bigyabai 1y agoIt really depends, the metrics are kinda all over the place right now: https://docs.google.com/spreadsheets/d/1SF1u0J2vJ-ou-R_Ry1JZQ0iscOZL8UKHpdVFr85tNLU/edit?gid=0#gid=0 https://docs.google.com/spreadsheets/d/1SF1u0J2vJ-ou-R_Ry1JZ... (cited from https://lmsys.org/blog/2025-10-13-nvidia-dgx-spark/ https://lmsys.org/blog/2025-10-13-nvidia-dgx-spark/)
- vardump 1y ago3060 doesn't have 128 GB RAM.
- moffkalast 1y ago128GB / 12 GB = ~11, * 200€ = only 2200€ plus mining rig mobo. It would be cheaper to buy up a dozen 3060s and build a custom PC around them than to buy the Spark.
- pjmlp 1y agoExcept the Spark was designed to have everything nicely working.
- Mars008 1y agoIt will be really hard to put a dozen in a single PC. Then to connect them at good speed. Add to that a few new power lines to feed it all.
- moffkalast 1y agoTrue, and overkill too. With DDR5 partial offloading it would only take probably two or three at most to outperform it in every metric except power draw. My point was more that the pricing is absurd for the performance.
- Mars008 1y agoI agree on pricing. But it includes 'free' software. Similar model Apple has. You don't buy just hardware.
- suprjami 1y agoAnd a 14B model running at 22tg/s means you won't be using that 128G RAM for inference either.
- yunohn 1y agoYeah I’m honestly unclear on Nvidia’s thinking here - inference speed is unbelievably slow for the price. Given the extreme advantage they have with CUDA and the whole AI/ML ecosystem, barely matching Apple’s M-ultra speeds is a choice…
- airspresso 1y agoDefinitely a choice to give it low memory bandwidth. Probably to avoid customers thinking it can replace any data center GPU for inference use-cases.
- kgeist 1y agoYou can use all 128 GB if you use a MoE model