4 ms·
> Given the choice between having a team of 3 people working under me or 1000 dedicated H100s for my work, I’d have to think for a little bit. But the exact nu
by Jabbles 3y ago
> Given the choice between having a team of 3 people working under me or 1000 dedicated H100s for my work, I’d have to think for a little bit.
But the exact numbers are important. An H100 is ~$2/hour [1], so 1000 is $16M/year (24/7). Even if Google gets a massive internal discount that's still way more than 3 people's total cost. If you have to choose between 16 (highly paid, senior) people or 1000 H100s would you have to think about the choice?
Then again when you revisit this comment in a few years' time the original comparison may be correct.
[1] https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-pricing/ https://gpus.llm-utils.org/h100-gpu-cloud-availability-and-p...
- silvestrov 3y agopurchase price != sales price. Very unlikely that Google pays $16M for a single H100. Amazon has it at $44k: https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphics/dp/B0C3XH4QSJ https://www.amazon.com/Tesla-NVIDIA-Learning-Compute-Graphic...
- inexcf 3y agoBetter reread the post. It was about renting GPUs. 16M for 1000 H100 per year. So i guess 44k at Amazon would make that 44M Dollar.
- ExoticPearTree 3y agoI don’t think the author meant for this to be treated as an absolute, more like food for thought. And if you really insist on 24/7 comparison, you would come to need about 12 people, as the expected productivity per person is of about 6 hrs/day. Factor in the fact that people need vacations, sick days, weekends off… it looks like 1000 H100 might actually be a good trade-off in the very near future.