2 ms·

> ...we are already looking at dropping $100k on hardware to run local models... Just think how much further that $100K would have gone if the hardware market

> ...we are already looking at dropping $100k on hardware to run local models... Just think how much further that $100K would have gone if the hardware market wasn't so screwed-up. Anecdote: I priced-out adding 1TB of RAM to a four node cluster a couple months ago. The cluster was purchased in fall of 2024 w/ 4 nodes, each with 256GB RAM. The nodes cost just over $14K apiece back in 2024 (entire box, not just the RAM). Dell wanted >$90K a couple months ago to add 256GB to each node.

cyberax 5mo ago
> Dell wanted >$90K a couple months ago to add 256GB to each node. RAM is expensive, but not THAT expensive. I just bought 128Gb for about $5k for our build cluster (it's not even for AI, sigh). Even if you need larger-sized DIMM sticks, it's still going to be in the vicinity of ~15k tops.
- deleted 5mo ago
  [deleted]
- EvanAnderson 5mo ago
  It was crazy. I found the part on the open market for a lot less but the edict from the Customer was to buy from Dell to keep the support entitlement intact. That inflated the price to an astronomical level to be sure. I haven't had problems w/ Dell support and 3rd party memory, personally, but given the machines' application I understood the concern.