3 ms·

S3 API has become lingua franca and is supported by open source (minio), storage vendors (QNAP), as well as plugins that translate S3 API calls to APIs for comp

by slt2021 3y ago

S3 API has become lingua franca and is supported by open source (minio), storage vendors (QNAP), as well as plugins that translate S3 API calls to APIs for competing cloud providers (s3proxy). all this is done because S3 provides unmatched durability and reliability at a dirt cheap cost of $22/terabyte/month of storage (with the first 50Tb/mo free!). Try to beat that reliability guarantees with whatever you handrolled, and I bet you will never be able to beat the cost of S3, even match the durability, reliability, availability guarantees at any reasonable cost at all from https://aws.amazon.com/s3/storage-classes/ https://aws.amazon.com/s3/storage-classes/: Key Features: Low latency and high throughput performance Designed for durability of 99.999999999% of objects across multiple Availability Zones have you ever built anything with 11 nines? (as in eleven nines)

herbstein 3y ago
> S3 API has become lingua franca S3 API support sounds great until your costumer builds a system with an "S3 compatible object storage" product. Soon you discover that many "S3 compatible" solutions aren't actually that compatible when pushed.
- slt2021 3y ago
  get-object and put-object is really all you need. everything else is nice to have
baz00 3y ago
S3 is fine until you want your data to leave AWS. Then it costs $92 / TB to get it out again. Also S3 has durability guarantees but it's very difficult to do a durable transactional write to S3. Try it a few million times and see. The API is a defacto shitty standard. These two facts are rather interesting when it comes to doing a restore from your supposed backup or wonder why consistency guarantees between external metadata services (DB) and what is in S3 don't always line up.
- slt2021 3y ago
  and why would you ever take raw data out of AWS ? if it is for migration: it is one time cost that anyone can swallow easily if they decided to leave AWS for something else. If your data is worth < $94/tb - it is really not worth pulling it out of AWS. Just let it sit there. or just use cloudfront to download your data ($8.5/Tb)?
  - jameshush 3y ago
    On top of that, if it's a big enough deal, most salespeople at other cloud providers (GCP, Oracle etc.) will gladly pay you to migrate. They'd probably even throw one of their Solutions Engineers at the problem and do it for you for free.
    - baz00 3y ago
      Selling your soul to a different crack dealer. Hmm.
      - catchnear4321 3y ago
        crackheads will move mountains for a score. this checks out.

herbstein 3y ago

> S3 API has become lingua franca S3 API support sounds great until your costumer builds a system with an "S3 compatible object storage" product. Soon you discover that many "S3 compatible" solutions aren't actually that compatible when pushed.

slt2021 3y ago
get-object and put-object is really all you need. everything else is nice to have

baz00 3y ago

S3 is fine until you want your data to leave AWS. Then it costs $92 / TB to get it out again. Also S3 has durability guarantees but it's very difficult to do a durable transactional write to S3. Try it a few million times and see. The API is a defacto shitty standard. These two facts are rather interesting when it comes to doing a restore from your supposed backup or wonder why consistency guarantees between external metadata services (DB) and what is in S3 don't always line up.

slt2021 3y ago
and why would you ever take raw data out of AWS ? if it is for migration: it is one time cost that anyone can swallow easily if they decided to leave AWS for something else. If your data is worth < $94/tb - it is really not worth pulling it out of AWS. Just let it sit there. or just use cloudfront to download your data ($8.5/Tb)?
- jameshush 3y ago
  On top of that, if it's a big enough deal, most salespeople at other cloud providers (GCP, Oracle etc.) will gladly pay you to migrate. They'd probably even throw one of their Solutions Engineers at the problem and do it for you for free.
  - baz00 3y ago
    Selling your soul to a different crack dealer. Hmm.
    - catchnear4321 3y ago
      crackheads will move mountains for a score. this checks out.