5 ms·

Glacier pricing has to be the most convoluted AWS pricing structure and can really screw you. Google Nearline is a much better option IMO. Seconds of retrieval

Glacier pricing has to be the most convoluted AWS pricing structure and can really screw you. Google Nearline is a much better option IMO. Seconds of retrieval time and still the same low price, and much easier to calculate your costs when looking into large downloads. https://cloud.google.com/storage/docs/nearline?hl=en https://cloud.google.com/storage/docs/nearline?hl=en

_g9ex 11y ago
There's also Backblaze B2 (public beta): https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html Their pricing is great (0.5c/month) but I'm a little worried about their single DC.
- andmarios 11y ago
  3 days ago they sent a newsletter about their new (alpha quality) b2sync tool which essentially is a “rsync to backblaze” utility. This makes their offer very interesting.
  - josephagoss 11y ago
    Can this tool offer deduplication (so that changing a folder name does not re-upload thousands of files) or is that something I would have to code into my own backup solution?
    - andmarios 11y ago
      No, it is even less than rsync. It can only upload a folder recursively and skip files with common modification time on local and remote. Considering though that before that you couldn't even upload a directory, only files, this is a huge step. :)
      - BorisMelnik 11y ago
        I currently need some help whipping up a solution for this.
        manquer 11y ago
        git with lfs
- wdr1 11y ago
  I hear people say Backblaze is a single DC frequently, but I've never been able to find anything on their site confirming or denying. Is there any source for the single DC? EDIT: Nevermind. I did some more googling & found one: https://www.backblaze.com/blog/b2-cloud-storage-frequent-questions/ https://www.backblaze.com/blog/b2-cloud-storage-frequent-que...
- fencepost 11y ago
  They may also take a while to have the same level of software support, since their API isn't compatible with the S3 API. I know of at least one online backup software provider that works with multiple storage backends but which when asked did not intend to support B2 (since Backblaze is technically a competitor).
onethumb 11y ago
They're both difficult to grok but for different reasons. (Well, the same underlying reason - bandwidth - just accounted for differently). Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs. Nearline is difficult to grok because it simply caps your bandwidth at 4 MB/s per TB stored. It starts at 1 MB/s. Basically, both are optimized to make retrieval slow and/or expensive because the cost optimizations they're doing internally are incentivizing very cold data, very big data, or both.
- boulos 11y ago
  Note that with "on-demand I/O" you can basically remove this I/O throttle (best effort though) at an increased cost. Nearline is still cheaper than dealing with glacier (or even S3 Infrequent Access) particularly for small files. Disclaimer: I work on Compute Engine (GCE) but not Cloud Storage (GCS).
- vacri 11y ago
  >Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs. It does go against their slogan "only pay for what you use", though. You're not using that 15GB/hour for 744 hours, and haven't explicitly selected provisioned IOPs.
- bonzini 11y ago
  > It starts at 1 MB/s. Not a problem all you can get is a crappy 8 Mbps ADSL. Oh well, let me whine... ;)
deleted 11y ago
[deleted]
- res0nat0r 11y ago
  Yes, there are Debian, Ubuntu, Centos, SUSE, CoreOS, RHEL and Windows Server images available for launch. You should see them in your cloud dashboard as choices when you go to launch an instance.
- TheDong 11y ago
  Entering exactly the phrase "Does Google cloud have any plain VMs (running CentOS for example) available" into Google search results in, as a first result, a link explaining exactly how to launch one of those [0]. Your comment doesn't really relate to the parent, to this thread and is a question that is trivially easy to answer. There's no point in hundreds-to-thousands of HN readers seeing your comment which can be answered in under a second and anyone who's used GCE already knows the answer to. Please considering querying Google, or a relevant knowledge base, before querying a tangentially related HN comment chain. [0]: https://cloud.google.com/compute/docs/operating-systems/linux-os https://cloud.google.com/compute/docs/operating-systems/linu...
  - derekp7 11y ago
    You're absolutely right, my apologies. -- I recall looking for this quite a while ago without success, but I should have done a fresh search before posting.
- boulos 11y ago
  Yes, Compute Engine is what you're after (https://cloud.google.com/compute/ https://cloud.google.com/compute/). Those predefined things are getting started / "click to deploy" for people that don't want to bother. Disclaimer: I work on Compute Engine.
boulos 11y ago
I'd point out that we're generally trying to offer simple, competitive pricing across our services (we're not totally there yet). This write up is a pretty good summary of our differences on the compute side: https://news.ycombinator.com/item?id=10922105 https://news.ycombinator.com/item?id=10922105 Disclaimer: I work on Compute Engine, and specific to that article launched Preemptible VMs.
- skrause 11y ago
  My main problem with the Google Cloud Storage as an individual customer in Germany is this: https://support.google.com/cloud/answer/6090602?hl=en https://support.google.com/cloud/answer/6090602?hl=en Since I can't find a way to set my account status to "personal" I'm somehow personally responsible to pay VAT (I wouldn't even know how to do that) if I don't want to risk getting problems with our version of the IRS. That doesn't sound like a nice way to do my personal backups. On AWS that's not a problem, Amazon bills and pays the VAT for me.
  - christop 11y ago
    As nonsensical as it is, you're not meant to be using Google Cloud Platform as an individual user at all. From that same page: "Google Cloud Platform services can be used only for business purposes in the European Union."
    - djhworld 11y ago
      Seems like AWS has capitalised on the gap to target the hobbyist AND enterprise markets, whereas google doesn't really want that.
  - boulos 11y ago
    Sorry. That's all any of us on Google Cloud can usefully say about the damn VAT issue. We've got people working on it, but it's a Google wide issue (and I honestly don't understand the last mile enough to know why we don't do exactly what you describe AWS doing). As a side reply to the poster below, it's not true that we're trying to exclude individuals. There are lots of happy, individual customers all over the world. We unfortunately just didn't get out ahead of this personal service / VAT thing as a company. So again, Sorry! And someone is working on it, but don't expect any immediate fixes when taxes/regulations/laws are involved.
skrause 11y ago
I think the best option for backups at AWS these days is the "Infrequent Access" storage class introduced a few months ago (probably as a reaction to Nearline): https://aws.amazon.com/blogs/aws/aws-storage-update-new-lower-cost-s3-storage-option-glacier-price-reduction/ https://aws.amazon.com/blogs/aws/aws-storage-update-new-lowe... It's almost as cheap as Glacier, but requires no waiting and has no complicated hidden costs, just simply somewhat higher request pricing, a minimum 30 days of storage and an extra $0.01 per GB for data retrievals.
- boulos 11y ago
  Sadly, there's still the "small file gotcha": > Standard - IA has a minimum object size of 128KB. Smaller objects will be charged for 128KB of storage. Rounding up all small files to 128 KB can be a huge deal. I for example use Nearline to directly "rsync" my NAS box for offsite backup (yeah I know, I should use something that turns it into a real archive or something, but I'm lazy and Synology has this built in). If those hundreds of thousands of (often) small files were rounded up, S3-IA would easily be more expensive than S3/GCS. Disclaimer: I work at Google on Compute Engine and am a happy Nearline customer myself.
  - abrookewood 11y ago
    You could solve that by using tar/zip to bundle your backups, but obviously that's an extra step.
  - fpoling 11y ago
    You can also try the duplicity backup tool. It uses tar files to store the primary backup and its binary incremental deltas. It supports a number of backup back-ends, but you can also just generate the backup to a local disk and then rsync it to a remote location.
  - rasz_pl 11y ago
    imo only crazy people upload anything to the cloud without compression+encryption.

_g9ex 11y ago

There's also Backblaze B2 (public beta): https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html Their pricing is great (0.5c/month) but I'm a little worried about their single DC.

andmarios 11y ago
3 days ago they sent a newsletter about their new (alpha quality) b2sync tool which essentially is a “rsync to backblaze” utility. This makes their offer very interesting.
- josephagoss 11y ago
  Can this tool offer deduplication (so that changing a folder name does not re-upload thousands of files) or is that something I would have to code into my own backup solution?
  - andmarios 11y ago
    No, it is even less than rsync. It can only upload a folder recursively and skip files with common modification time on local and remote. Considering though that before that you couldn't even upload a directory, only files, this is a huge step. :)
    - BorisMelnik 11y ago
      I currently need some help whipping up a solution for this.
      - manquer 11y ago
        git with lfs
wdr1 11y ago
I hear people say Backblaze is a single DC frequently, but I've never been able to find anything on their site confirming or denying. Is there any source for the single DC? EDIT: Nevermind. I did some more googling & found one: https://www.backblaze.com/blog/b2-cloud-storage-frequent-questions/ https://www.backblaze.com/blog/b2-cloud-storage-frequent-que...
fencepost 11y ago
They may also take a while to have the same level of software support, since their API isn't compatible with the S3 API. I know of at least one online backup software provider that works with multiple storage backends but which when asked did not intend to support B2 (since Backblaze is technically a competitor).

onethumb 11y ago

They're both difficult to grok but for different reasons. (Well, the same underlying reason - bandwidth - just accounted for differently). Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs. Nearline is difficult to grok because it simply caps your bandwidth at 4 MB/s per TB stored. It starts at 1 MB/s. Basically, both are optimized to make retrieval slow and/or expensive because the cost optimizations they're doing internally are incentivizing very cold data, very big data, or both.

boulos 11y ago
Note that with "on-demand I/O" you can basically remove this I/O throttle (best effort though) at an increased cost. Nearline is still cheaper than dealing with glacier (or even S3 Infrequent Access) particularly for small files. Disclaimer: I work on Compute Engine (GCE) but not Cloud Storage (GCS).
vacri 11y ago
>Glacier, as the article points out, is difficult to grok because there are provisioned bandwidth costs. It does go against their slogan "only pay for what you use", though. You're not using that 15GB/hour for 744 hours, and haven't explicitly selected provisioned IOPs.
bonzini 11y ago
> It starts at 1 MB/s. Not a problem all you can get is a crappy 8 Mbps ADSL. Oh well, let me whine... ;)

deleted 11y ago

[deleted]

res0nat0r 11y ago
Yes, there are Debian, Ubuntu, Centos, SUSE, CoreOS, RHEL and Windows Server images available for launch. You should see them in your cloud dashboard as choices when you go to launch an instance.
TheDong 11y ago
Entering exactly the phrase "Does Google cloud have any plain VMs (running CentOS for example) available" into Google search results in, as a first result, a link explaining exactly how to launch one of those [0]. Your comment doesn't really relate to the parent, to this thread and is a question that is trivially easy to answer. There's no point in hundreds-to-thousands of HN readers seeing your comment which can be answered in under a second and anyone who's used GCE already knows the answer to. Please considering querying Google, or a relevant knowledge base, before querying a tangentially related HN comment chain. [0]: https://cloud.google.com/compute/docs/operating-systems/linux-os https://cloud.google.com/compute/docs/operating-systems/linu...
- derekp7 11y ago
  You're absolutely right, my apologies. -- I recall looking for this quite a while ago without success, but I should have done a fresh search before posting.
boulos 11y ago
Yes, Compute Engine is what you're after (https://cloud.google.com/compute/ https://cloud.google.com/compute/). Those predefined things are getting started / "click to deploy" for people that don't want to bother. Disclaimer: I work on Compute Engine.

boulos 11y ago

I'd point out that we're generally trying to offer simple, competitive pricing across our services (we're not totally there yet). This write up is a pretty good summary of our differences on the compute side: https://news.ycombinator.com/item?id=10922105 https://news.ycombinator.com/item?id=10922105 Disclaimer: I work on Compute Engine, and specific to that article launched Preemptible VMs.

skrause 11y ago
My main problem with the Google Cloud Storage as an individual customer in Germany is this: https://support.google.com/cloud/answer/6090602?hl=en https://support.google.com/cloud/answer/6090602?hl=en Since I can't find a way to set my account status to "personal" I'm somehow personally responsible to pay VAT (I wouldn't even know how to do that) if I don't want to risk getting problems with our version of the IRS. That doesn't sound like a nice way to do my personal backups. On AWS that's not a problem, Amazon bills and pays the VAT for me.
- christop 11y ago
  As nonsensical as it is, you're not meant to be using Google Cloud Platform as an individual user at all. From that same page: "Google Cloud Platform services can be used only for business purposes in the European Union."
  - djhworld 11y ago
    Seems like AWS has capitalised on the gap to target the hobbyist AND enterprise markets, whereas google doesn't really want that.
- boulos 11y ago
  Sorry. That's all any of us on Google Cloud can usefully say about the damn VAT issue. We've got people working on it, but it's a Google wide issue (and I honestly don't understand the last mile enough to know why we don't do exactly what you describe AWS doing). As a side reply to the poster below, it's not true that we're trying to exclude individuals. There are lots of happy, individual customers all over the world. We unfortunately just didn't get out ahead of this personal service / VAT thing as a company. So again, Sorry! And someone is working on it, but don't expect any immediate fixes when taxes/regulations/laws are involved.

skrause 11y ago

I think the best option for backups at AWS these days is the "Infrequent Access" storage class introduced a few months ago (probably as a reaction to Nearline): https://aws.amazon.com/blogs/aws/aws-storage-update-new-lower-cost-s3-storage-option-glacier-price-reduction/ https://aws.amazon.com/blogs/aws/aws-storage-update-new-lowe... It's almost as cheap as Glacier, but requires no waiting and has no complicated hidden costs, just simply somewhat higher request pricing, a minimum 30 days of storage and an extra $0.01 per GB for data retrievals.

boulos 11y ago
Sadly, there's still the "small file gotcha": > Standard - IA has a minimum object size of 128KB. Smaller objects will be charged for 128KB of storage. Rounding up all small files to 128 KB can be a huge deal. I for example use Nearline to directly "rsync" my NAS box for offsite backup (yeah I know, I should use something that turns it into a real archive or something, but I'm lazy and Synology has this built in). If those hundreds of thousands of (often) small files were rounded up, S3-IA would easily be more expensive than S3/GCS. Disclaimer: I work at Google on Compute Engine and am a happy Nearline customer myself.
- abrookewood 11y ago
  You could solve that by using tar/zip to bundle your backups, but obviously that's an extra step.
- fpoling 11y ago
  You can also try the duplicity backup tool. It uses tar files to store the primary backup and its binary incremental deltas. It supports a number of backup back-ends, but you can also just generate the backup to a local disk and then rsync it to a remote location.
- rasz_pl 11y ago
  imo only crazy people upload anything to the cloud without compression+encryption.