8 ms·

How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b

by fuddle 3y ago

ZiiS 3y ago
They may have to pay a premium to secure ~¼ of the output; certainly unlikely to be that steep a discount.
- theptip 3y ago
  Semi analysis posted recently noting that Meta locked in these purchases a while ago; something like a year or more. So they probably didn’t pay today’s spot rate.
YetAnotherNick 3y ago
> $3.5b Which is a fourth of what they spent in VR/AR in a year. And Gen AI is something they could easily get more revenue as it has now become proven technology, and Meta could possibly leapfrog others because of the data moat.
- NBJack 3y ago
  What moat exactly? Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. I'm not even sure they can use historical data. Meta certainly has an edge in engineer count, undoubtedly. But I'd say they really, really want the metaverse to succeed more to have their on walled garden (i.e. equivalent power of Apple and Google stores, etc.). There's a reason they gave a hard pass to a Google partnership.
  - Dr_Birdbrain 3y ago
    I think the raw text inside Facebook groups is at least as valuable as Reddit data. Even if demographics data is restricted under European law, the raw text of people interacting is quite valuable.
    - calvinmorrison 3y ago
      facebooks downfall will be their lock in. every other social media platform lets you view a public profile, discussion groups etc. it's all locked inside facebook.
    - verticalscaler 3y ago
      Indeed, my deranged auntie posting on FB is approximately as valuable as my ADHD/PTSD quaranteeny nephew redditing.
      - fragmede 3y ago
        That ignores all the user groups that are on Facebook. From apartment communities aka Nextdoor to grief support counseling to the mindfulness therapy groups, there’s a wealth of user comments a tad bit higher than Uncle John’s racist rants.
      - infecto 3y ago
        Ahhh you had posted some other negative META criticism that was not even factual. Your made-up narratives really do not paint the correct picture.
  - YetAnotherNick 3y ago
    > Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. Source would be appreciated, because this is opposite of obvious. Regulations against using public first party would be a big news and I haven't heard of anything like that. They use my data for recommending feed so why not for answering my question?
    - NBJack 3y ago
      Insufficient reasons for consent: https://iapp.org/news/a/10-takeaways-from-the-irish-dpc-decisions-on-meta/ https://iapp.org/news/a/10-takeaways-from-the-irish-dpc-deci... Decreased ability to 'mix' it: https://www.linkedin.com/pulse/ecpms-why-first-party-data-big-problem-andrew-wailes-abw1e https://www.linkedin.com/pulse/ecpms-why-first-party-data-bi... First party data alone can't tell you whether an ad resulted in a sale, unless you own the entire process on your platform. Contrast this with what Apple has via its app store; the fees do more than generate money.
  - agar 3y ago
    > There's a reason they gave a hard pass to a Google partnership. AIUI, Google required Meta to basically cede control of a partnered OS to them: "After years of not focusing on VR or doing anything to support our work in the space, Google has been pitching AndroidXR to partners and suggesting, incredibly, that WE are the ones threatening to fragment the ecosystem when they are the ones who plan to do exactly that. "We would love to partner with them. They could bring their apps to Quest today! They could bring the Play store (with its current economics for 2d apps) and add value to all their developers immediately, which is exactly the kind of open app ecosystem we want to see. We would be thrilled to have them. It would be a win for their developers and all consumers and we’ll keep pushing for it. "Instead, they want us to agree to restrictive terms that require us to give up our freedom to innovate and build better experiences for people and developers—we’ve seen this play out before and we think we can do better this time around." -- From Mark Bosworth
- dougb5 3y ago
  Proven technology, maybe, but proven product-market fit for the kinds of things Facebook is using it for? Their linked blog about AI features gives examples "AI stickers" and image editing... cool, but are these potential multi-billion dollar lifts to their existing business? I guess I'm skeptical it's worthwhile unless they're able to unseat ChatGPT with a market-leading general purpose assistant.
  - pests 3y ago
    I have a few group chats just that devolve into hours of sending stickers or image generation back and forth, lately we've been "writing a book together" with @Meta AI as the ghost writer, and while it utterly sucks, its been a hilarious shared experience. I don't think anyone else has gotten that group chat with AI thing so nailed.
    - TaylorAlexander 3y ago
      On the podcast TrashFuture, November Kelly recently described AI systems as “garbage dispensers” which is both a funny image (why would anyone make a garbage dispenser??) and an apt description. Certainly these tools have some utility, but there are a load of startups claiming to “democratize creativity” by allowing anyone to publish AI generated slop to major platforms. On the podcast this phrase was used during discussion of a website which lets you create AI generated music and push it to Spotify, a move which Spotify originally pushed back on but has now embraced. Garbage dispenser indeed.
  - YetAnotherNick 3y ago
    > unseat ChatGPT with a market-leading general purpose assistant. It's not impossible. The prediction from many(not that I believe it) is that over long run modelling tricks would become common knowledge and only thing that matters is compute and data, both of which Meta has. Also there could be a trend of LLMs for ads or feed recommendation in the future as they has large completely unstructured dataset per user across multiple sites.
    - cj 3y ago
trsohmers 3y ago
Significantly more than that; MFN pricing for NVIDIA DGX H100 (which has been getting priority supply allocation, so many have been suckered into buying them in order to get fast delivery) is ~$309k, while a basically equivalent HGX H100 system is ~$250k, coming to a price per GPU at the full server level being ~$31.5k. With Meta’s custom OCP systems integrating the SXM baseboards from NVIDIA, my guess is that their cost per GPU would be in the ~$23-$25k range.
- fuddle 3y ago
  350,000 NVIDIA H100 x $23k = $8b :0
  - verticalscaler 3y ago
    Wait till you find out how much they spent on VR. It is a real loophole in the economy. If you're a trillion dollar company the market will insist you set such sums on fire just to be in the race for $current-hype. If they do it drives their market cap higher still and if they don't they risk being considered un-innovative and therefore doomed to irrelevancy and the market cap will spiral downwards. Sort of reminds me of The Producers.
    - oblio 3y ago
      The thing is, this could be considered basic research, right? Basic research IS setting money on fire until (and if) that basic research turns into TCP/IP, Ethernet and the Internet.
      - verticalscaler 3y ago
        I wish. Funnily enough Arpanet and all that Xerox stuff were like <$50 million (inflation adjusted!) total. Some real forward thinkers were able to work the system by breaking off a tiny pittance of a much larger budget. Where as I think this more appropriately can be considered the meta PR budget. They simply can't not spend it, would look bad for Wall Street. Have to keep up with the herd.
dekhn 3y ago
That sounds like a reasonable budget for 3 years of hardware at a major AI company.
vineyardmike 3y ago
It’s often forgotten now, but just a few years NVidia was cancelled production batches and writing down inventory when the GPU shortage cleared. No one needed more GPUs. It also happens to be when Meta first announced they were going to increase CapEx spending on compute. I’m guessing that Meta got a sweetheart deal to help take a lot of inventory for NVidia and make commitments for future purchases.
- transcriptase 3y ago
  I don’t think it was that nobody needed GPUs. It was that nvidia thought they could get scalper margins by restricting supply after the shortage showed people were willing to pay scalper prices.
loeg 3y ago
Yes, billions in GPU cap ex.

How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b

by fuddle 3y ago

How much are they paying for H100's? If they are paying $10k: 350,000 NVIDIA H100 x $10k = $3.5b

ZiiS 3y ago

They may have to pay a premium to secure ~¼ of the output; certainly unlikely to be that steep a discount.

theptip 3y ago
Semi analysis posted recently noting that Meta locked in these purchases a while ago; something like a year or more. So they probably didn’t pay today’s spot rate.

YetAnotherNick 3y ago

> $3.5b Which is a fourth of what they spent in VR/AR in a year. And Gen AI is something they could easily get more revenue as it has now become proven technology, and Meta could possibly leapfrog others because of the data moat.

NBJack 3y ago
What moat exactly? Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. I'm not even sure they can use historical data. Meta certainly has an edge in engineer count, undoubtedly. But I'd say they really, really want the metaverse to succeed more to have their on walled garden (i.e. equivalent power of Apple and Google stores, etc.). There's a reason they gave a hard pass to a Google partnership.
- Dr_Birdbrain 3y ago
  I think the raw text inside Facebook groups is at least as valuable as Reddit data. Even if demographics data is restricted under European law, the raw text of people interacting is quite valuable.
  - calvinmorrison 3y ago
    facebooks downfall will be their lock in. every other social media platform lets you view a public profile, discussion groups etc. it's all locked inside facebook.
  - verticalscaler 3y ago
    Indeed, my deranged auntie posting on FB is approximately as valuable as my ADHD/PTSD quaranteeny nephew redditing.
    - fragmede 3y ago
      That ignores all the user groups that are on Facebook. From apartment communities aka Nextdoor to grief support counseling to the mindfulness therapy groups, there’s a wealth of user comments a tad bit higher than Uncle John’s racist rants.
    - infecto 3y ago
      Ahhh you had posted some other negative META criticism that was not even factual. Your made-up narratives really do not paint the correct picture.
- YetAnotherNick 3y ago
  > Much of the user data they have access to is drying up due to new regulations, some of which prohibit IIRC direct use on models as well. Source would be appreciated, because this is opposite of obvious. Regulations against using public first party would be a big news and I haven't heard of anything like that. They use my data for recommending feed so why not for answering my question?
  - NBJack 3y ago
    Insufficient reasons for consent: https://iapp.org/news/a/10-takeaways-from-the-irish-dpc-decisions-on-meta/ https://iapp.org/news/a/10-takeaways-from-the-irish-dpc-deci... Decreased ability to 'mix' it: https://www.linkedin.com/pulse/ecpms-why-first-party-data-big-problem-andrew-wailes-abw1e https://www.linkedin.com/pulse/ecpms-why-first-party-data-bi... First party data alone can't tell you whether an ad resulted in a sale, unless you own the entire process on your platform. Contrast this with what Apple has via its app store; the fees do more than generate money.
- agar 3y ago
  > There's a reason they gave a hard pass to a Google partnership. AIUI, Google required Meta to basically cede control of a partnered OS to them: "After years of not focusing on VR or doing anything to support our work in the space, Google has been pitching AndroidXR to partners and suggesting, incredibly, that WE are the ones threatening to fragment the ecosystem when they are the ones who plan to do exactly that. "We would love to partner with them. They could bring their apps to Quest today! They could bring the Play store (with its current economics for 2d apps) and add value to all their developers immediately, which is exactly the kind of open app ecosystem we want to see. We would be thrilled to have them. It would be a win for their developers and all consumers and we’ll keep pushing for it. "Instead, they want us to agree to restrictive terms that require us to give up our freedom to innovate and build better experiences for people and developers—we’ve seen this play out before and we think we can do better this time around." -- From Mark Bosworth
dougb5 3y ago
Proven technology, maybe, but proven product-market fit for the kinds of things Facebook is using it for? Their linked blog about AI features gives examples "AI stickers" and image editing... cool, but are these potential multi-billion dollar lifts to their existing business? I guess I'm skeptical it's worthwhile unless they're able to unseat ChatGPT with a market-leading general purpose assistant.
- pests 3y ago
  I have a few group chats just that devolve into hours of sending stickers or image generation back and forth, lately we've been "writing a book together" with @Meta AI as the ghost writer, and while it utterly sucks, its been a hilarious shared experience. I don't think anyone else has gotten that group chat with AI thing so nailed.
  - TaylorAlexander 3y ago
    On the podcast TrashFuture, November Kelly recently described AI systems as “garbage dispensers” which is both a funny image (why would anyone make a garbage dispenser??) and an apt description. Certainly these tools have some utility, but there are a load of startups claiming to “democratize creativity” by allowing anyone to publish AI generated slop to major platforms. On the podcast this phrase was used during discussion of a website which lets you create AI generated music and push it to Spotify, a move which Spotify originally pushed back on but has now embraced. Garbage dispenser indeed.
- YetAnotherNick 3y ago
  > unseat ChatGPT with a market-leading general purpose assistant. It's not impossible. The prediction from many(not that I believe it) is that over long run modelling tricks would become common knowledge and only thing that matters is compute and data, both of which Meta has. Also there could be a trend of LLMs for ads or feed recommendation in the future as they has large completely unstructured dataset per user across multiple sites.
  - cj 3y ago

trsohmers 3y ago

Significantly more than that; MFN pricing for NVIDIA DGX H100 (which has been getting priority supply allocation, so many have been suckered into buying them in order to get fast delivery) is ~$309k, while a basically equivalent HGX H100 system is ~$250k, coming to a price per GPU at the full server level being ~$31.5k. With Meta’s custom OCP systems integrating the SXM baseboards from NVIDIA, my guess is that their cost per GPU would be in the ~$23-$25k range.

fuddle 3y ago
350,000 NVIDIA H100 x $23k = $8b :0
- verticalscaler 3y ago
  Wait till you find out how much they spent on VR. It is a real loophole in the economy. If you're a trillion dollar company the market will insist you set such sums on fire just to be in the race for $current-hype. If they do it drives their market cap higher still and if they don't they risk being considered un-innovative and therefore doomed to irrelevancy and the market cap will spiral downwards. Sort of reminds me of The Producers.
  - oblio 3y ago
    The thing is, this could be considered basic research, right? Basic research IS setting money on fire until (and if) that basic research turns into TCP/IP, Ethernet and the Internet.
    - verticalscaler 3y ago
      I wish. Funnily enough Arpanet and all that Xerox stuff were like <$50 million (inflation adjusted!) total. Some real forward thinkers were able to work the system by breaking off a tiny pittance of a much larger budget. Where as I think this more appropriately can be considered the meta PR budget. They simply can't not spend it, would look bad for Wall Street. Have to keep up with the herd.

dekhn 3y ago

That sounds like a reasonable budget for 3 years of hardware at a major AI company.

vineyardmike 3y ago

It’s often forgotten now, but just a few years NVidia was cancelled production batches and writing down inventory when the GPU shortage cleared. No one needed more GPUs. It also happens to be when Meta first announced they were going to increase CapEx spending on compute. I’m guessing that Meta got a sweetheart deal to help take a lot of inventory for NVidia and make commitments for future purchases.

transcriptase 3y ago
I don’t think it was that nobody needed GPUs. It was that nvidia thought they could get scalper margins by restricting supply after the shortage showed people were willing to pay scalper prices.

loeg 3y ago

Yes, billions in GPU cap ex.