5 ms·
3060 doesn't have 128 GB RAM.
by vardump 1y ago
3060 doesn't have 128 GB RAM.
- moffkalast 1y ago128GB / 12 GB = ~11, * 200€ = only 2200€ plus mining rig mobo. It would be cheaper to buy up a dozen 3060s and build a custom PC around them than to buy the Spark.
- pjmlp 1y agoExcept the Spark was designed to have everything nicely working.
- Mars008 1y agoIt will be really hard to put a dozen in a single PC. Then to connect them at good speed. Add to that a few new power lines to feed it all.
- moffkalast 1y agoTrue, and overkill too. With DDR5 partial offloading it would only take probably two or three at most to outperform it in every metric except power draw. My point was more that the pricing is absurd for the performance.
- Mars008 1y agoI agree on pricing. But it includes 'free' software. Similar model Apple has. You don't buy just hardware.
- suprjami 1y agoAnd a 14B model running at 22tg/s means you won't be using that 128G RAM for inference either.
- yunohn 1y agoYeah I’m honestly unclear on Nvidia’s thinking here - inference speed is unbelievably slow for the price. Given the extreme advantage they have with CUDA and the whole AI/ML ecosystem, barely matching Apple’s M-ultra speeds is a choice…
- airspresso 1y agoDefinitely a choice to give it low memory bandwidth. Probably to avoid customers thinking it can replace any data center GPU for inference use-cases.
- kgeist 1y agoYou can use all 128 GB if you use a MoE model