6 ms·

Meta has their own data centers so they definitely didn't pay the equivalent to what AWS costs

by nuz 3y ago

freedomben 3y ago
Good point, although it's possible that with the extreme price of GPUs that it cost more to train by buying hardware than it would to rent. For example it might take two to three years before the GPUs are paid for by customers.
- samus 3y ago
  Since the GPUs can be rented out afterwards, they amortize very quickly with prices in the order of $1/h.
- fleischhauf 3y ago
  I think AWS prices scale with hardware price
- greenavocado 3y ago
  Linux reserved cost of p3.16xlarge is $146,362.0800 annually. On-demand cost is $214,444.8000 annually I am pretty damn sure I could build a 8 GPU Intel Xeon E5-2686 v4 (Broadwell) (that's what Amazon uses - it's $30 to $75 on eBay) server for less than that and come out ahead on electricity even at full throttle. RTX 4090 are just under $2000 each on eBay. 8 GPU × $2000 (RTX 4090) + $1000 (for the rest of the computer) = $17,000 If pulling 2kW continuously at 15 cents per kW*hr for 1 year that's 2000 watts × 365 days × (0.15/(kW×hr)) or $2,628 In total the computer will cost $19,628 if you throw it in the dumpster at the end of each calendar year of using it. If you stack internet cost of $200 a month on top, that's $2400 a year, which raises your annual cost to: $22,028 This is still $124,334 cheaper per year than one AWS 8-GPU server if you fully depreciate your own hardware at the end of year 1 to $0. I could hire an engineer in America to babysit it with the money left over.
  - ghshephard 3y ago
    An A100-80 GPU goes for about $20K each.
    - greenavocado 3y ago
      The instances in question use Tesla V100-SXM2-16GB
  - OtherShrezzing 3y ago
    Are consumer grade RTX 4090 cards going to be suitable for running full tilt 24/7 for a year? Those things are fine to stress on the latest game for a few hours at a time, but would probably cause some defects from significant heat stress after just a few days at 100%. This is inconsequential when you're playing Overwatch for a few hours a night and a frame drops now and again. If you're training an iteratively developed LLM though, physical defects could propagate into huge deficiencies in the final model.
    - _ea1k 3y ago
      I don't think they'd become a fire hazard, but it is true that one would likely pick something else for this application. Having said that, switching to something like the Tesla V100-SXM2-16GB wouldn't cost that much more. TBH, I'm shocked at how many people treat Amazon as the first choice for this stuff. Much of it isn't even what most would consider a "production" workload. You are paying for a lot of enterprise-readiness that you don't need for training.
      - pdntspa 3y ago
        > TBH, I'm shocked at how many people treat Amazon as the first choice for this stuff You can thank Amazon's legions of salespeople for that, particularly the end of year junket in Las Vegas where attendees are so pampered that about the only thing they won't do is suck your dick Oh, yeah, they'll also yell at you on stage if you complain about their UI
      - robrenaud 3y ago
- deleted 3y ago
  [deleted]
packetslave 3y ago
Meta also doesn't pay AWS anywhere near retail price for instances.
- KMnO4 3y ago
  Why is this the case? Even AWS internal pays the same AWS prices as everyone else
  - elcomet 3y ago
    They have their own data centers, they don't use AWS
    - packetslave 3y ago
      They don't use AWS to run the big three apps (FB, Insta, WhatsApp). They very much use them for other things. FB data center machines tend to be highly-specialized and optimized for running the apps, not general-purpose compute.
  - vidarh 3y ago
    I'm less surprised if AWS internally pays AWS list prices, because that's just internal accounting. From the even relatively small AWS customers I know, none of them needed to get very far into the 6 digits per year spend before a couple of quiet mentions to their account manager that they were reviewing other options was enough to get steep discounts. Add in lots of credits, and if you pay list price, you're being taken to the cleaners.. I've done contract work for clients to be ready to migrate both as part of maximising credits and as part of negotiating posture, and the savings can be enormous (though it'd still usually be cheaper to use managed servers).
  - packetslave 3y ago
    Couple of reasons: Meta's annual spend with AWS is large enough that they'll have a negotiated blanket discount that takes a fixed percentage off the top of their monthly spend. This is very common for larger AWS customers, not just Meta. For instances specifically, any planned usage will be using either reserved instances or at minimum a compute savings plan (CSP) that drops the hourly rate dramatically in exchange for a committed number of instance hours, with or without an upfront payment. Finally, there may be a negotiated rate for specific instance types built into the contract. Again, common for very large customers. source: I was on one of the cloud-related infrastructure teams (left in early 2022). I have no idea about their spend (or discounts) today, but two years ago it was enough that Andy Jassy would meet 1:1 with Mark to "discuss the relationship".

Meta has their own data centers so they definitely didn't pay the equivalent to what AWS costs

by nuz 3y ago

Meta has their own data centers so they definitely didn't pay the equivalent to what AWS costs

Good point, although it's possible that with the extreme price of GPUs that it cost more to train by buying hardware than it would to rent. For example it might take two to three years before the GPUs are paid for by customers.

samus 3y ago
Since the GPUs can be rented out afterwards, they amortize very quickly with prices in the order of $1/h.
fleischhauf 3y ago
I think AWS prices scale with hardware price
greenavocado 3y ago
Linux reserved cost of p3.16xlarge is $146,362.0800 annually. On-demand cost is $214,444.8000 annually I am pretty damn sure I could build a 8 GPU Intel Xeon E5-2686 v4 (Broadwell) (that's what Amazon uses - it's $30 to $75 on eBay) server for less than that and come out ahead on electricity even at full throttle. RTX 4090 are just under $2000 each on eBay. 8 GPU × $2000 (RTX 4090) + $1000 (for the rest of the computer) = $17,000 If pulling 2kW continuously at 15 cents per kW*hr for 1 year that's 2000 watts × 365 days × (0.15/(kW×hr)) or $2,628 In total the computer will cost $19,628 if you throw it in the dumpster at the end of each calendar year of using it. If you stack internet cost of $200 a month on top, that's $2400 a year, which raises your annual cost to: $22,028 This is still $124,334 cheaper per year than one AWS 8-GPU server if you fully depreciate your own hardware at the end of year 1 to $0. I could hire an engineer in America to babysit it with the money left over.
- ghshephard 3y ago
  An A100-80 GPU goes for about $20K each.
  - greenavocado 3y ago
    The instances in question use Tesla V100-SXM2-16GB
- OtherShrezzing 3y ago
  Are consumer grade RTX 4090 cards going to be suitable for running full tilt 24/7 for a year? Those things are fine to stress on the latest game for a few hours at a time, but would probably cause some defects from significant heat stress after just a few days at 100%. This is inconsequential when you're playing Overwatch for a few hours a night and a frame drops now and again. If you're training an iteratively developed LLM though, physical defects could propagate into huge deficiencies in the final model.
  - _ea1k 3y ago
    I don't think they'd become a fire hazard, but it is true that one would likely pick something else for this application. Having said that, switching to something like the Tesla V100-SXM2-16GB wouldn't cost that much more. TBH, I'm shocked at how many people treat Amazon as the first choice for this stuff. Much of it isn't even what most would consider a "production" workload. You are paying for a lot of enterprise-readiness that you don't need for training.
    - pdntspa 3y ago
      > TBH, I'm shocked at how many people treat Amazon as the first choice for this stuff You can thank Amazon's legions of salespeople for that, particularly the end of year junket in Las Vegas where attendees are so pampered that about the only thing they won't do is suck your dick Oh, yeah, they'll also yell at you on stage if you complain about their UI
    - robrenaud 3y ago
deleted 3y ago
[deleted]

packetslave 3y ago

Meta also doesn't pay AWS anywhere near retail price for instances.

KMnO4 3y ago
Why is this the case? Even AWS internal pays the same AWS prices as everyone else
- elcomet 3y ago
  They have their own data centers, they don't use AWS
  - packetslave 3y ago
    They don't use AWS to run the big three apps (FB, Insta, WhatsApp). They very much use them for other things. FB data center machines tend to be highly-specialized and optimized for running the apps, not general-purpose compute.
- vidarh 3y ago
  I'm less surprised if AWS internally pays AWS list prices, because that's just internal accounting. From the even relatively small AWS customers I know, none of them needed to get very far into the 6 digits per year spend before a couple of quiet mentions to their account manager that they were reviewing other options was enough to get steep discounts. Add in lots of credits, and if you pay list price, you're being taken to the cleaners.. I've done contract work for clients to be ready to migrate both as part of maximising credits and as part of negotiating posture, and the savings can be enormous (though it'd still usually be cheaper to use managed servers).
- packetslave 3y ago
  Couple of reasons: Meta's annual spend with AWS is large enough that they'll have a negotiated blanket discount that takes a fixed percentage off the top of their monthly spend. This is very common for larger AWS customers, not just Meta. For instances specifically, any planned usage will be using either reserved instances or at minimum a compute savings plan (CSP) that drops the hourly rate dramatically in exchange for a committed number of instance hours, with or without an upfront payment. Finally, there may be a negotiated rate for specific instance types built into the contract. Again, common for very large customers. source: I was on one of the cloud-related infrastructure teams (left in early 2022). I have no idea about their spend (or discounts) today, but two years ago it was enough that Andy Jassy would meet 1:1 with Mark to "discuss the relationship".