3 ms·
I literally wrote about running quantized models and how much more affordable it could be in the very next sentence. Please don't reply if you can't be bothered
by anigbrowl 4mo ago
I literally wrote about running quantized models and how much more affordable it could be in the very next sentence. Please don't reply if you can't be bothered to read the entire comment, it's not that long.
- sosodev 4mo agoI read the comment, thanks. I just disagree with your cost estimate. Even for a small business that needs high throughput they could probably do it for far less than $300k if they aren’t just blindly buying the first big nvidia setup they can.
- anigbrowl 4mo agoBut your estimate was explicitly based on running the 4-bit version. The full uncompressed Deepseek R1 model is 1.4TB. You'd need 16 h100s for this, and considering the new price for an 8 node cluster is about $350k, I think my estimate of $300k was pretty generous, assuming you'd buy used or liquidation or something. https://www.gmicloud.ai/en/blog/2026-cost-of-renting-or-uying-nvidia-h100-gpus-for-data-centers https://www.gmicloud.ai/en/blog/2026-cost-of-renting-or-uyin...