5 ms·
Meta buys 600k H100s to train LLaMa3
- satya71 3y agoI doubt they're paying retail prices like the tweet implies. Still can't be a small number if that is true.
- dist-epoch 3y agoIf there is a discount, it will be tiny: > “We’re so short on GPUs the less people use our products the better… We’d love it if they use it less because we don’t have enough GPUs” — Sam Altman > Meta who has the 2nd most H100 GPUs in the world is increasingly leveraging GPU supply as a recruiting strategy. https://www.datagravity.dev/p/2023-year-in-review-the-great-gpu https://www.datagravity.dev/p/2023-year-in-review-the-great-...
- yuvalsteuer 3y agoEXACTLY! what I was assuming.
- samspenc 3y agoLooks like it retails for $30K, I won't be surprised if the $20K is the bulk price they are paying, so the numbers could be correct. Though as a wise man [1] once said: "The more you buy, the more you save". [1] Jensen Huang, NVidia's CEO, says this quite often at conferences, its become a meme at this point, one example https://twitter.com/NVIDIAAI/status/917681161391841280 https://twitter.com/NVIDIAAI/status/917681161391841280
- RecycledEle 3y agoI should have built a video card company based on VAX designs.
- yuvalsteuer 3y agodo it now.
- sylware 3y agoHow many "parameters" for the LLaMa3?
- yuvalsteuer 3y agozuck won't puck.
- throwaway888abc 3y agohttps://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal) https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nomi... lol, that's an investment!
- samspenc 3y ago$7 billion is a lot, but not comparable to the GDPs of countries which are in the trillions (unless you compare to the smallest of the long-tail). Meta's annual revenues alone exceed $100B+ at this point, assuming these purchases are staggered over 2-3 years, it is basically adding a 2-3% overhead to their expenses on an annualized basis.
- gnabgib 3y agoWrong (it's 350K according to Zuck on insta) and dupe of "Meta will have more than 350.000 Nvidia H100s this year"[0], or "Zuckerberg's Meta Is Spending Billions to Buy 350k Nvidia H100 GPUs"[1] [0]: https://news.ycombinator.com/item?id=39044911 https://news.ycombinator.com/item?id=39044911 [1]: https://news.ycombinator.com/item?id=39046193 https://news.ycombinator.com/item?id=39046193
- yuvalsteuer 3y ago350k in new H100s and in total 600k in H100s equivalents.
- samspenc 3y agoI wonder what he means by "H100 equivalents" - are these GPUs built in-house or from competitors like AMD / Intel that give equivalent performance to Nvidia's H100s?
- fotcorn 3y agoI wouldn't be surprised if the answer is "all of the above" and more. Not only is there limited availability of NVIDIA chips, it's also useful to tell Jensen that you actually could buy from somewhere else, even if you don't really want to. At the end of the day, the bottleneck is TSMC, on which all GPUs are produced right now, even the ones from Intel.
- mrkeen 3y agoI can't wait to not be able to run the trained model.