3 ms·
Depends how many 3090s you have
by teej 9mo ago
Depends how many 3090s you have
- woggy 9mo agoHow many do you need to run inference for 1 user on a model like Opus 4.5?
- ronsor 9mo ago8x 3090. Actually better make it 8x 5090. Or 8x RTX PRO 6000.
- worldsavior 9mo agoHow is there enough space in this world for all these GPUs
- Forgeties79 9mo agoMilk crates and fans, baby. Party like it’s 2012.
- filoleg 9mo agoJust try calculating how many RTX 5090 GPUs by volume would fit in a rectangular bounding box of a small sedan car, and you will understand how. Honda Civic (2026) sedan has 184.8” (L) × 70.9” (W) × 55.7” (H) dimensions for an exterior bounding box. Volume of that would be ~12,000 liters. An RTX 5090 GPU is 304mm × 137mm, with roughly 40mm of thickness for a typical 2-slot reference/FE model. This would make the bounding box of ~1.67 liters. Do the math, and you will discover that a single Honda Civic would be an equivalent of ~7,180 RTX 5090 GPUs by volume. And that’s a small sedan, which is significantly smaller than an average or a median car on the US roads.
- worldsavior 9mo agoWhat about what's around the GPU? Motherboard etc.
- filoleg 9mo agoI didn’t do the napkin math on it earlier, because I don’t believe it really matters for making the point I was making. I don’t care about looking up real numbers, so I will just overestimate heavily. Let’s say that for a large enough number of GPUs, the overhead of all the surrounding equipment would be around 20% (amortized). So you can just take the number of GPUs I calculated in my previous comment, multiply by 0.8, and you get your answer.
- worldsavior 9mo agoThis is not 20% , it's 100%+.
- antonvs 9mo agoNow factor in power and cooling...
- reactordev 9mo agoDon’t forget to lease out idle time to your neighbors for credits per 1M tokens…
- adastra22 9mo ago48x 3090’s actually.
- _flux 9mo agoNone, if you have time to wait, and a bit of memory on the computer.