4 ms·

I wonder how much they pay for 100PB in AWS, depending on the storage service/tier. Also it will be quite expensive if they ever want to move out of AWS. Or is

I wonder how much they pay for 100PB in AWS, depending on the storage service/tier. Also it will be quite expensive if they ever want to move out of AWS. Or is a project like this considered to be "eternal"?

beeeeerp 3y ago
It’s actually hard to say. The prices these companies pay can be very different than list prices for us mortals. It’s one of my biggest complaints about AWS; the prices seem high, but in reality they are very opaque.
- woleium 3y ago
  they probably pay around 40% of list, depending on how much their ARR is.
- andersa 3y ago
  This really needs to be made illegal already. All prices should be public and you should not be allowed to charge different customers different prices for the same automated service. I should be able to store my data at the exact same $/TB price as Epic Games, for example. They once mentioned it costs them less than a penny to serve several GB from AWS - why the fuck is it more expensive for me?
  - rescbr 3y ago
    If you are willing to sign a contract and commit to spend $ millions per year on AWS services for 5+ years like Epic Games did, then sure, you can get the same $/TB price they got. At some point of very large usage, you also have to work with AWS teams to give input on your demand forecast. Unless you have such large usage you get list price, that also gives you the illusion of infinite elasticity. This is not exclusive to AWS though, all hyperscalers have this policy.
  - tnmom 3y ago
    Because you’re a more expensive customer per dollar of revenue, statistically - there are constant costs per customer for support, billing, etc.
    - andersa 3y ago
      Billing is automated. What support? S3 doesn't care whether it serves a PB of Epic traffic or from thousands of small customers, it's all automated. There is no actually reasonable explanation for the service being 10-100 times more expensive than greed.
      - tnmom 3y ago
        Cards get declined. Support tickets get created. Customers die and you have to hold all their stuff open for 90 days while trying to reach them, without a chance of getting paid. You might not be stupid, but there are people who look _just like you_ who will email support over a six cent billing issue which turns out to be correct anyway - but just burned a couple hours of support people’s time. Epic probably doesn’t do stuff like that.
  - maxerickson 3y ago
    Would it also be illegal to have 2 very similar services, with no small tier on one of them?
  - scarface_74 3y ago
    Every business negotiates volume discounts. Do you think consumer goods companies charge Walmart for products at the same rate that they charge a mom and pop shop? I’m sure if you had the same volume as Netflix you would pay less than the retail price to. If you ever flew on a plane, I can guarantee you that there are probably 20+ different prices being charged for the same seat.
  - greggsy 3y ago
    Many jurisdictions do require prices to be clearly displayed for goods, but services are a different beast. You have limited resources to deliver these services, customer needs can be complex, and differ across regions.
  - belter 3y ago
    So what you are saying is that if I buy one Tesla, and you buy one hundred Tesla cars, they are not authorized to give you a discount?
  - quickthrowman 3y ago
    > They once mentioned it costs them less than a penny to serve several GB from AWS - why the fuck is it more expensive for me? You spend far less money than a corporation, and there are customer costs beyond the actual bandwidth/compute charges.
  - hodgesrm 3y ago
    > This really needs to be made illegal already. All prices should be public and you should not be allowed to charge different customers different prices for the same automated service. Why? Different people get different prices for products all the time. It's fundamental to a functioning free market.
  - dekhn 3y ago
    Why should it be illegal for service providers to negotiate prices?
woleium 3y ago
The EU has forced the providers to allow a “final egress on account closing” at no cost (or i should say, they are offering it at no cost). GCP was first in January [0], the others have since matched. 0: https://www.theregister.com/AMP/2024/01/11/google_cloud_egress_fees/ https://www.theregister.com/AMP/2024/01/11/google_cloud_egre...
- jl6 3y ago
  Presumably doesn't apply to the cold archive tiers where a great deal of the cost is retrieval/egress?
  - sp332 3y ago
    It's also worth noting that the migration service only applies to data stored in one of the following services: BigQuery, Cloud Bigtable, Cloud SQL, Cloud Storage, Datastore, Filestore, Spanner, and Persistent Disk.
    - 0cf8612b2e1e 3y ago
      Still seems you would save a bundle if you had to pay to promote the data from archival to hot and then do the free exodus.
  - woleium 3y ago
    the article lists the following for gcp: BigQuery, Cloud Bigtable, Cloud SQL, Cloud Storage, Datastore, Filestore, Spanner, and Persistent Disk.
binarymax 3y ago
Using the calculator (https://calculator.aws/#/createCalculator/S3 https://calculator.aws/#/createCalculator/S3), AWS S3 "infrequent access" is $1.3M per month for 100PB. Not sure if that's single zone.
- Uehreka 3y ago
  If you put it in Glacier Deep Archive it’s closer to $1.3M per year, just to set a lower bound.
  - vidarh 3y ago
    Deep Archive has retrieval times up to 12 hours, so for on-prem/colo that is then competing with buying a few tape robots (you can get a single tape archive that handles >1 exabyte, but you'll of course want redundancy without having to have staff shuffling tape in an archive that will run into the thousands). You can buy management software that tries to replicate a glacier-like experience for things like SpectraLogic tape libraries but they may not be the cheapest option. It ends up becoming a very different beast than the storage classes w/fast enough retrieval guarantees to need disk, in any case.
- greggsy 3y ago
  Unlikely that they want it hot or even warm. That data almost definitely went to Gkacier while they worked out what to do with it.
  - xyzzy_plugh 3y ago
    You overestimate the enterprise.

beeeeerp 3y ago

It’s actually hard to say. The prices these companies pay can be very different than list prices for us mortals. It’s one of my biggest complaints about AWS; the prices seem high, but in reality they are very opaque.

woleium 3y ago
they probably pay around 40% of list, depending on how much their ARR is.
andersa 3y ago
This really needs to be made illegal already. All prices should be public and you should not be allowed to charge different customers different prices for the same automated service. I should be able to store my data at the exact same $/TB price as Epic Games, for example. They once mentioned it costs them less than a penny to serve several GB from AWS - why the fuck is it more expensive for me?
- rescbr 3y ago
  If you are willing to sign a contract and commit to spend $ millions per year on AWS services for 5+ years like Epic Games did, then sure, you can get the same $/TB price they got. At some point of very large usage, you also have to work with AWS teams to give input on your demand forecast. Unless you have such large usage you get list price, that also gives you the illusion of infinite elasticity. This is not exclusive to AWS though, all hyperscalers have this policy.
- tnmom 3y ago
  Because you’re a more expensive customer per dollar of revenue, statistically - there are constant costs per customer for support, billing, etc.
  - andersa 3y ago
    Billing is automated. What support? S3 doesn't care whether it serves a PB of Epic traffic or from thousands of small customers, it's all automated. There is no actually reasonable explanation for the service being 10-100 times more expensive than greed.
    - tnmom 3y ago
      Cards get declined. Support tickets get created. Customers die and you have to hold all their stuff open for 90 days while trying to reach them, without a chance of getting paid. You might not be stupid, but there are people who look _just like you_ who will email support over a six cent billing issue which turns out to be correct anyway - but just burned a couple hours of support people’s time. Epic probably doesn’t do stuff like that.
- maxerickson 3y ago
  Would it also be illegal to have 2 very similar services, with no small tier on one of them?
- scarface_74 3y ago
  Every business negotiates volume discounts. Do you think consumer goods companies charge Walmart for products at the same rate that they charge a mom and pop shop? I’m sure if you had the same volume as Netflix you would pay less than the retail price to. If you ever flew on a plane, I can guarantee you that there are probably 20+ different prices being charged for the same seat.
- greggsy 3y ago
  Many jurisdictions do require prices to be clearly displayed for goods, but services are a different beast. You have limited resources to deliver these services, customer needs can be complex, and differ across regions.
- belter 3y ago
  So what you are saying is that if I buy one Tesla, and you buy one hundred Tesla cars, they are not authorized to give you a discount?
- quickthrowman 3y ago
  > They once mentioned it costs them less than a penny to serve several GB from AWS - why the fuck is it more expensive for me? You spend far less money than a corporation, and there are customer costs beyond the actual bandwidth/compute charges.
- hodgesrm 3y ago
  > This really needs to be made illegal already. All prices should be public and you should not be allowed to charge different customers different prices for the same automated service. Why? Different people get different prices for products all the time. It's fundamental to a functioning free market.
- dekhn 3y ago
  Why should it be illegal for service providers to negotiate prices?

woleium 3y ago

The EU has forced the providers to allow a “final egress on account closing” at no cost (or i should say, they are offering it at no cost). GCP was first in January [0], the others have since matched. 0: https://www.theregister.com/AMP/2024/01/11/google_cloud_egress_fees/ https://www.theregister.com/AMP/2024/01/11/google_cloud_egre...

jl6 3y ago
Presumably doesn't apply to the cold archive tiers where a great deal of the cost is retrieval/egress?
- sp332 3y ago
  It's also worth noting that the migration service only applies to data stored in one of the following services: BigQuery, Cloud Bigtable, Cloud SQL, Cloud Storage, Datastore, Filestore, Spanner, and Persistent Disk.
  - 0cf8612b2e1e 3y ago
    Still seems you would save a bundle if you had to pay to promote the data from archival to hot and then do the free exodus.
- woleium 3y ago
  the article lists the following for gcp: BigQuery, Cloud Bigtable, Cloud SQL, Cloud Storage, Datastore, Filestore, Spanner, and Persistent Disk.

binarymax 3y ago

Using the calculator (https://calculator.aws/#/createCalculator/S3 https://calculator.aws/#/createCalculator/S3), AWS S3 "infrequent access" is $1.3M per month for 100PB. Not sure if that's single zone.

Uehreka 3y ago
If you put it in Glacier Deep Archive it’s closer to $1.3M per year, just to set a lower bound.
- vidarh 3y ago
  Deep Archive has retrieval times up to 12 hours, so for on-prem/colo that is then competing with buying a few tape robots (you can get a single tape archive that handles >1 exabyte, but you'll of course want redundancy without having to have staff shuffling tape in an archive that will run into the thousands). You can buy management software that tries to replicate a glacier-like experience for things like SpectraLogic tape libraries but they may not be the cheapest option. It ends up becoming a very different beast than the storage classes w/fast enough retrieval guarantees to need disk, in any case.
greggsy 3y ago
Unlikely that they want it hot or even warm. That data almost definitely went to Gkacier while they worked out what to do with it.
- xyzzy_plugh 3y ago
  You overestimate the enterprise.