6 ms·

Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10

Delivering "HDD prices" is a bold claim there. They charge $20/TB/month for basic cloud storage. You can build storage servers for $20/TB flat. If you add 10% for local parity, 15% free space, 5% in spare drives, and $2000/rack/month overhead, then triple everything for redundancy purposes, then over a 3 year period the price of using your own hard drives is $115/TB and google's price is $720. Over 5 years it's $145 versus $1200. And that's before they charge you massive bandwidth fees.

saagarjha 2y ago
You forgot paying yourself to set that up.
- Dylan16807 2y ago
  That's covered by the build and overhead numbers. But if you want more on the build side, an extra $10k of labor per rack of 9 servers only increases the cost per TB by about $4.
- concerndc1tizen 2y ago
  You're paying the same for "cloud engineers". Also, don't forget the hidden cost/risk of giving a third party full access to your data.
  - Sonnigeszeug 2y ago
    Clicking yourself a Bucket takes 5 Minutes. Building a Server and keeping it secure and up-to-date and fixing hardware issues, takes relevant time
    - coredog64 2y ago
      Not to mention that I can: - Create a bucket and store 1MB in it without any overhead - Create 50 buckets with strong perimeters around them such that someone deleting the entire account doesn’t bring down the other 49 - Create a bucket and fill it with terabytes of data within seconds and don’t need to wait for hardware to be racked and stacked - Create a bucket, fill it with 2TB of data, and delete it tomorrow Cloud is more than bare metal, but plenty of folks discount the cost benefits of elasticity.
      - concerndc1tizen 2y ago
        I suspect the problem is that we're engineers in domains that have very different needs. For example, I agree that elasticity is great. But at the same time, to me, it sounds like bad engineering. Why do you need to store terabytes of data and then delete it - couldn't it be processed continuously, streamed, compressed, process changes only, and so on. A lot of engineering today is incredibly wasteful. Maybe your data source doesn't care, and just provides you with terabyte csv files, and you have no choice, but for engineers that care about efficiency, it reeks. It might make a lot of sense in a highly corporate context where everything is hard, nobody cares, and the cost of inefficiency is just passed on to the customer (i.e. often government and tax payers). But the real problem here is that customers aren't demanding more efficiency.
        Sonnigeszeug 2y ago
        Alone the fact of audit gives you a lot of reasons to keep data. Even if it gets downsampled one way or the other. And plenty of use cases have natural growth. I do not throw away my pictures for example. Data also grows dependent of users. More users, more 'live' data. We have such a huge advantage with digital, we need to stop thinking its wasteful. Everything we do digital (pictures, finance data, etc.) is so much more energy and space efficient than what we had 20 years ago, we should just not delete data because we feel its wasteful.
Lex-2008 2y ago
I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a list of these numbers :) Moreover, these are not even the cheapest one. The cheapest one I found had prices starting from $6.5 for 5TB, going down to $0.64/TB/month for plans starting with 25TB (called uloz, but I haven't tested them yet). Also, looking at lowendbox you can find a VPS in Canada with 2TB storage for $5/month and run whatever you want there. How all that compares to $20/TB/month?! Please feel free to correct me if i'm comparing apples to oranges, though. But I can't believe all of these offers are scam or so-called "promotional" offers which cost companies more than you pay for it.
- ddorian43 2y ago
  Of what you mentioned, only backblaze is similar (object storage with S3-like API), all others are apples to oranges.
  - Dylan16807 2y ago
    You don't need very many terabytes to cover the labor cost of installing and maintaining an S3-compatible server program.
    - ddorian43 2y ago
      You need a very big cluster for it to be worth it though for non-backup use-cases when using HDDs.
- ksec 2y ago
  Thank You. So backblaze for $6/TB a month. I could have a TB of Data backed up safely against file corruption? I wonder how have I missed that. Now you could use it with Synology NAS and it is a lot cheaper than doing RAID 5 for ZFS / BTRFS with Muti redundancy. I wonder if there are any NAS that does that automatically? Any drawbacks? Also wonder if the price could go down to $5 / TB in a few years time.
Twirrim 2y ago
Leverage erasure encoding for durability and avoid both the tripling and local parity. You'll get better durability than 3x while only taking up significantly less than 2x the space Backblaze open sourced their library and talk about it here, https://www.backblaze.com/blog/Reed-Solomon https://www.backblaze.com/blog/Reed-Solomon. They use a 17:20 ratio that'll get them 3 drive failure resistance for just 1.17x stretch (ie a 100mb file gets that resilient while taking up 117mb of space)

saagarjha 2y ago

You forgot paying yourself to set that up.

Dylan16807 2y ago
That's covered by the build and overhead numbers. But if you want more on the build side, an extra $10k of labor per rack of 9 servers only increases the cost per TB by about $4.
concerndc1tizen 2y ago
You're paying the same for "cloud engineers". Also, don't forget the hidden cost/risk of giving a third party full access to your data.
- Sonnigeszeug 2y ago
  Clicking yourself a Bucket takes 5 Minutes. Building a Server and keeping it secure and up-to-date and fixing hardware issues, takes relevant time
  - coredog64 2y ago
    Not to mention that I can: - Create a bucket and store 1MB in it without any overhead - Create 50 buckets with strong perimeters around them such that someone deleting the entire account doesn’t bring down the other 49 - Create a bucket and fill it with terabytes of data within seconds and don’t need to wait for hardware to be racked and stacked - Create a bucket, fill it with 2TB of data, and delete it tomorrow Cloud is more than bare metal, but plenty of folks discount the cost benefits of elasticity.
    - concerndc1tizen 2y ago
      I suspect the problem is that we're engineers in domains that have very different needs. For example, I agree that elasticity is great. But at the same time, to me, it sounds like bad engineering. Why do you need to store terabytes of data and then delete it - couldn't it be processed continuously, streamed, compressed, process changes only, and so on. A lot of engineering today is incredibly wasteful. Maybe your data source doesn't care, and just provides you with terabyte csv files, and you have no choice, but for engineers that care about efficiency, it reeks. It might make a lot of sense in a highly corporate context where everything is hard, nobody cares, and the cost of inefficiency is just passed on to the customer (i.e. often government and tax payers). But the real problem here is that customers aren't demanding more efficiency.
      - Sonnigeszeug 2y ago
        Alone the fact of audit gives you a lot of reasons to keep data. Even if it gets downsampled one way or the other. And plenty of use cases have natural growth. I do not throw away my pictures for example. Data also grows dependent of users. More users, more 'live' data. We have such a huge advantage with digital, we need to stop thinking its wasteful. Everything we do digital (pictures, finance data, etc.) is so much more energy and space efficient than what we had 20 years ago, we should just not delete data because we feel its wasteful.

Lex-2008 2y ago

I like your comparison with self-built storage, but comparing $20/TB/month with other CLOUD offerings, we see: * hetzner storage box starts from $4/month for 1TB, and then goes down to $2.4/TB/month if you rent a 10TB box. * mega starts from €10/month for 2TB, and goes down to €2/TB/month if you get a 16TB plan * backblaze costs (starts from?) $6/TB/month I was looking for a cheap cloud storage recently, so have a a list of these numbers :) Moreover, these are not even the cheapest one. The cheapest one I found had prices starting from $6.5 for 5TB, going down to $0.64/TB/month for plans starting with 25TB (called uloz, but I haven't tested them yet). Also, looking at lowendbox you can find a VPS in Canada with 2TB storage for $5/month and run whatever you want there. How all that compares to $20/TB/month?! Please feel free to correct me if i'm comparing apples to oranges, though. But I can't believe all of these offers are scam or so-called "promotional" offers which cost companies more than you pay for it.

ddorian43 2y ago
Of what you mentioned, only backblaze is similar (object storage with S3-like API), all others are apples to oranges.
- Dylan16807 2y ago
  You don't need very many terabytes to cover the labor cost of installing and maintaining an S3-compatible server program.
  - ddorian43 2y ago
    You need a very big cluster for it to be worth it though for non-backup use-cases when using HDDs.
ksec 2y ago
Thank You. So backblaze for $6/TB a month. I could have a TB of Data backed up safely against file corruption? I wonder how have I missed that. Now you could use it with Synology NAS and it is a lot cheaper than doing RAID 5 for ZFS / BTRFS with Muti redundancy. I wonder if there are any NAS that does that automatically? Any drawbacks? Also wonder if the price could go down to $5 / TB in a few years time.

Twirrim 2y ago

Leverage erasure encoding for durability and avoid both the tripling and local parity. You'll get better durability than 3x while only taking up significantly less than 2x the space Backblaze open sourced their library and talk about it here, https://www.backblaze.com/blog/Reed-Solomon https://www.backblaze.com/blog/Reed-Solomon. They use a 17:20 ratio that'll get them 3 drive failure resistance for just 1.17x stretch (ie a 100mb file gets that resilient while taking up 117mb of space)