21 ms·

Amazon S3 and Glacier Price Reductions

293 points by jeffbarr 10y ago

QUFB 10y ago
I currently use S3 Infrequent Access buckets for some personal projects. These Glacier price reductions, along with the much better retrieval model look really great. However using Glacier as a simple store from the command-line seems horribly convoluted: https://docs.aws.amazon.com/cli/latest/userguide/cli-using-glacier.html https://docs.aws.amazon.com/cli/latest/userguide/cli-using-g... Does anyone know of any good tooling around Glacier for the command line?
- jinl 10y ago
  Perhaps you can script in python using boto3.
- res0nat0r 10y ago
  If you don't want to mess with all of that, you could use the standard "aws s3" command to upload your files to your s3 bucket like normal, and just apply an archive policy to your bucket or archive/ prefix or whatnot and it will automatically transfer your files to glacier for you.
  - phonon 10y ago
    Yup, works great.
Alex3917 10y ago
While I'm not going to complain about a price reduction, I'd honestly be more excited if S3 implemented support for additional headers and redirect rules. Right now, anyone hosting a single page app (e.g. Angular/React) behind S3 and Cloudfront is going to get an F on securityheaders.io. And even worse, there is no way to prerender an SPA site for search engines without standing up an nginx proxy on ec2, which completely eliminates almost all of the benefits from Cloudfront. This is because right now S3 can only redirect based on a key prefix or error code, not based on a user agent like Googlebot or whatever. This means that even if you can technically drop a <meta name="fragment" content="!"> tag in your front end and then have S3 redirect on the key prefix '?_escaped_fragment_=', that will be a 301 redirect. This means that Google will ignore any <link rel="canonical" href="..."> tag on the prerendered page and will instead index https://api.yoursite.com https://api.yoursite.com or wherever your prerendered content is being hosted rather than your actual site. Not only is it a bunch of extra work to stand up an nginx proxy as a workaround, but it's also a whole extra set of security concerns, scaling concerns, etc. Not a good situation. edit: For more info on the prerendering issues, c.f.: https://github.com/prerender/prerender/issues/93 https://github.com/prerender/prerender/issues/93 https://gist.github.com/thoop/8165802 https://gist.github.com/thoop/8165802
- ponyfleisch 10y ago
  If i'm not wrong, cloudfront actually follows 302 redirects internally instead of passing the redirect on to the client. but since ?_escaped_fragment_= is a suffix, not a prefix, i don't think redirection rules help.
  - Alex3917 10y ago
    Confusingly, _escaped_fragment_= is always a prefix if you have a #! in your url, otherwise I think it can come anywhere in the URL params.[1] Here is the exact system of redirects we were using, along with an explanation of why it didn't work: https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving-prerendered-s3-sites https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving... [1] https://prerender.io/js-seo/angularjs-seo-get-your-site-indexed-and-to-the-top-of-the-search-results/ https://prerender.io/js-seo/angularjs-seo-get-your-site-inde...
    - ponyfleisch 10y ago
      Ah, yes, if you use #!. In that case, can't you do a cloudfront custom behaviour on /?_escaped_fragment_=*? I havent't tried this.
- tbrock 10y ago
  Hmm, as someone who is about to move from using a webserver to hosting our single page app on S3 you may have just convinced me not to. What exactly are the benefits over simply setting up nginx if not simplicity? Yeah, it's great to just serve the asset from s3 but the complexity of what you just described negates it almost entirely.
  - Alex3917 10y ago
    > What exactly are the benefits over simply setting up nginx if not simplicity? We weren't fully aware of these limitations when we decided to host our site on S3. Had we been, we may have just used nginx. There are obviously a ton of benefits to S3 and Cloudfront, it's just than in practice you can't really get them if you need Google to index your site. And while Google claims they can now execute javascript and include async content, in practice this isn't true for any real Angular or React site. And even if every search engine were to magically execute js correctly, you'd still need to prerender your site in order for Facebook and Twitter to populate the preview cards for your site with the proper headline, summary, and image.
cpkpad 10y ago
Well, the costs are nicer, but mostly, Glacier goes from an unusable pricing model to a usable one. I was terrified to use Glacier. The previous model, if you made requests too rapidly, you might be hit with thousands of dollars of bills for relatively small data retrievals -- very easy to make a very expensive bug. I had wanted Amazon to wrap it in something where they managed that complexity for a long time. Looks like they finally did. Now the only thing Amazon needs to do is expand free tiers on all of their services, or at least very low cost ones. I prototype a lot of things from home for work -- kinda 20% time style projects where I couldn't really budget resources for it. The free tier is great for that. All services ought to have it -- especially RDS. I ought to be able to have a slice of a database (even kilobytes/tens of accesses/not-guaranteed security/shared server) paying nothing or pennies.
- Johnny555 10y ago
  A t2.large costs 10 cents/hour, a t2.medium RDS instance costs 7 cents/hour. If you put in 50 hours/month on this side project, that's $8.50 for compute plus maybe $3 for ~ 30GB of storage. $11.50/month doesn't sound too hard to budget for.
  - andrioni 10y ago
    It might be more about bureaucracy than about cost. At my last job even small expenses would require printing forms and getting the CFO to sign them, so in the end it was pretty much not worth it for small tests.
    - brianwawok 10y ago
      Your work can't give you a few VMs in a lab somewhere that you can get free rein to prototype on? That is usually not too hard to get.. it seems the idea of the Amazon free tier is to give people a taste of AWS so they can decide to go in more or not. It's not really designed to be a free prototype for existing large customers product. Like the other poster said, you can host a tiny VM for $6 or so a month, not a big expense. If you are asking for $4000 a month production cluster, yes that is harder to just get..
scrollaway 10y ago
Anyone else finding their S3 bill consisting of mostly PUT/COPY/POST/LIST queries? Our service has a ton of data going in, very little going out and we're sitting with 95% of the bill being P/C/P/L queries and only the remaining 5% being storage. Either way, good news on the storage price reductions :)
- cyberferret 10y ago
  What app/site are you using to upload to S3? I use a combination of CloudBerry Backup and Arq Backup on my Macs/PCs here and the requests aren't that high (on average about 30Gb of data per machine in around 300K files). I am guessing it comes down to the algorithm used to compare and upload/download files. I believe the two solutions above use a separate 'index' file on S3 to track file compares.
  - scrollaway 10y ago
    It's more that we have a pretty high throughput system, using Lambda. Users authenticate with an API gateway endpoint, we do a PUT to store a descriptor file, send a presigned PUT URL back so they can upload their file, we then process the file and do a COPY+DELETE to move it out of the "not yet processed" stage and finally do another PUT to upload the resulting processed file. Despite a lot of data, the storage bill is barely scratching $40, but we're at almost $700/mo on API calls.
    - cyberferret 10y ago
      Ah, thanks for the extra info. We have several web apps that take user uploads and store it to S3 buckets here too, but still we don't see an adversely high request load. Not sure if the handshaking involved in getting a pre-signed URL will be upping your count? We just use the AWS SDK on our Ruby back end. The user file is first uploaded to the (EC2) app server, then we use the SDK call to transfer it to the S3 bucket. Our storage and request costs are about equal at this stage. Using Lambda/Node, I guess that the SDK is not an option and you have to use the pre-signed URL method? Or else use Python and the SDK library?
sfdksfjkasfas 10y ago
43434
user5994461 10y ago
That's both a good and a terrible change. - The price reduction on S3 is good! Kudos AWS. - The price change on glacier is a fucking disaster. They replaced the _single_ expensive glacier fee with the choice among 3 user selectable fee models (Standard, Expedited, Bulk). It's an absolute nightmare added on top of the current nightmare (e.g. try to understand the disks specifications & pricing. It takes months of learning). I cannot follow the changes, too complicated. I cannot train my devs to understand glacier either, too much of a mess. AWS if you read this: Please make your offers and your pricing simpler, NEVER more complicated. (Even a single pricing option would be significantly better than that, even if its more expensive.)
- tw04 10y ago
  Simple is easy to shop around. Their goal is not to make pricing simple, don't hold your breath.
- Lazare 10y ago
  I believe this HUGELY simplifies glacier pricing. I don't think you understand just how insanely, incredibly, bizarrely complicated the old glacier pricing was.
  - user5994461 10y ago
    There are 3 pricing models for S3, the 3rd one (glacier) having 3 sub-pricing models to be chosen at time of requests. I don't think you realize how insanely complex the entire S3 pricing model once you get out of the "standard price". Maybe I just have too much empathy for my poor devs and ops who try to understand how much what they're doing is gonna cost them. It's only one full page, both sided, of text after all.
    - Lazare 10y ago
      I'm not saying that S3 isn't complicated, or that the pricing model, even after the changes, isn't nuts. I'm saying that anyone who thinks this is more complicated than it was does not understand just how crazy glacier pricing was before. Three static glacier pricing tiers is a lot better than the previously system which is so complex that earlier versions of the AWS pricing page just gave up and called it "free". (Briefly: The old Glacier model's pricing wasn't based around data transfer, but on your automatically retroactively provisioned data transfer capacity based on your peak data transfer rate, billed on a sliding scale as if you'd maintained that rate for the entire month. There's probably a less intuitive way to bill people for downloading data, but if so I've never seen it. It was a system seemingly designed to prevent users from knowing what any given retrieval would cost.)
physcab 10y ago
This a really dumb question, but since I've never used Glacier what does the workflow for a Glacier application look like? I'm used to the world of immediate access needs, and fast API responses, so I can't imagine sending off a request to an API with a response "Your data will be ready in 1-5 hours, come back later".
- needcaffeine 10y ago
  It may be for backing up infrequently accessed data (compliance logs, etc) for example. Hypothetical: you create a logging service for users to send all their log data to you. You promise 365 days of archives, but 30 days of data accessible at any time. You create a lifecycle rule on your S3 bucket to automatically archive data to Glacier 30 days after creation. On the 31st day, your user decides they want to look at an old log. They click the big Download button. You display a message saying they'll get an email from you when that data is ready to download.
- scrollaway 10y ago
  Glacier is not for data you want readily available. It's for when you care more about storage than access.
  - pauloday 10y ago
    Right, but if you want to retrieve some data through their API, how does it work? Normally you open the connection, ask for the data, then receive it and close the connection, does that change if there's a 5+ hour wait between the ask and the receive? Do you just leave the connection open? Provide them with a webhook to call when it's ready? I don't personally care about the answer but I'm pretty sure that's what they were asking.
    - extra88 10y ago
      At work our needs are simple, we manually run the aws cli to sync files up to S3 where there's a 1 day lifecycle policy to move them to Glacier. We don't use the API for restores, we do those through the web console and check back in a few hours to see if the files are downloadable. I think through the API you do not leave the connection open, you check with whatever frequency you want and when it's ready, the response will include the temporary location on S3 for the file.
jaytaylor 10y ago
EDIT: My mistake, this is the new S3 pricing! NOT Glacier pricing! Thank you res0nat0r. Am I understanding this right? $0.023/GB/month for Glacier, so * 12 months/year = $0.276/GB/year, which means: 10GB = $2.70/year 100GB = $27.00/year 1TB = $270.00/year ... And this is only the storage cost. This doesn't take into account the cost should you actually decise to retrieve the data. So considering a 1TB hard drive [0] costs $50.00, how is this cost effective? I can buy 5x1TB hard drives for the price of 1TB on Glacier. I understand there is overhead to managing it yourself. So, is this just not targeted to technically proficient folks? [0] https://www.amazon.com/Blue-Cache-Desktop-Drive-WD10EZEX/dp/B0088PUEPK https://www.amazon.com/Blue-Cache-Desktop-Drive-WD10EZEX/dp/...
- acdha 10y ago
  S3 data is backed up in three geographically separated data-centers, so you need to budget for 3 copies. It's a running service (rather than cold storage) which means you need servers, power network ports, etc. Again, times 3. Finally, include the software development to build the server stack and the staff for 24x7 ops and security. It's possible to beat their pricing but most people who actually do manage it by cutting in an area which they don't need.
- eli 10y ago
  Where are you going to keep it? On site? Not so good for disaster recovery.
  - deleted 10y ago
    [deleted]
- derekdahmer 10y ago
  That's comparing apples and oranges - your hard drive doesn't live in a secure, fireproof data center in the cloud you can access from anywhere.
  - jeffbarr 10y ago
    Or eggs and omelettes...
Perceptes 10y ago
Is anyone using either S3 or Glacier to store encrypted backups of their personal computer(s)? I've only used Time Machine to back up my machine for a long time, but I don't really trust it and would like to have another back up in the cloud. Any tools that automate back up and restore to/from S3/Glacier? What are your experiences?
- res0nat0r 10y ago
  Check out Duplicity or Duplicati. Also Arq for OSX I believe is very nice and has a great GUI but isn't free.
  - jonatron 10y ago
    +1 for Duplicity
- icecreammatt 10y ago
  I use Arq (https://www.arqbackup.com/ https://www.arqbackup.com/) and it works very well. I've only tested retrieving small amounts of data from it so I can't comment much on a large bill. I only wish it worked on Linux. I've been thinking about seeing if it would work with Wine.
  - chrislund 10y ago
    Ditto. I've been using Arq for 4 years now to back up nearly a TB of data to S3 and Google Cloud Storage.
    - CoachRufus87 10y ago
      Out of curiosity, why both?
      - chrislund 10y ago
        I guess mostly to not have all my cloud backup eggs in one basket.
msravi 10y ago
> For example, retrieving 500 archives that are 1 GB each would cost 500GB x $0.01 + 500 x $0.05/1,000 = $5.25 Shouldn't that be $5.025? Or did I misunderstand?
la6470 10y ago
This is just awesome ... AWS continue to surprise with new features.. now only if I could mark a bucket encrypted so anything uploaded automatically gets encrypted!!
ww520 10y ago
Is the outgoing bandwidth still the same price? Bandwidth cost is kind of high compared to other services.
- dx034 10y ago
  Yes, they are high for all 3 major cloud providers. I guess it's to keep you from using the cheapest part of each provider. That way you have to use their whole ecosystem and they can offer some products below costs to lure people in.
DanBlake 10y ago
Would really like to see some massive reductions in the operation costs and most importantly, bandwidth costs. The bandwidth costs are so far out of line with what the network transfer actually costs, it just feels like price fixing between the major cloud players that nobody is drastically reducing those prices, only storage prices. Charging 5 cents per gigabyte (at their maximum published discount level) is the equivalent to paying $16,000 per month for a 1 gigabit line. This does not count any operation costs either, which could add thousands in cost as well, depending on how you are using S3. There are several providers that offer a unmetered 1gbps line PLUS a dedicated server for ~600-750/mo. Providers like OVH offer the bandwidth for as little as 100/month. ( https://www.ovh.com/us/dedicated-servers/bandwidth-upgrade.xml https://www.ovh.com/us/dedicated-servers/bandwidth-upgrade.x... ) I am just not sure how amazon can justify a 160x price increase over OVH or a 30x increase over dedicated server + transfer. For the time being, the best bet is to use S3 for your storage and then have a heavily caching non amazon CDN on top of it (like cloudflare) to save the ridiculous bandwidth costs.
- Scoundreller 10y ago
  I use s3 as a webserver for static pages. Total bandwidth costs: 17 cents a month. 14 or so cents now. And I have some tens of thousands of visits per month. The only cognitive thought I've had to put in since setup was questioning one aspect of a bill (it amounted to pennies, but I couldn't figure it out, and maybe they were overbilling bigger entities). And they gave me a month free (including DNS costs). I wish the Route53 would go down... Hard to beat, even if I were running job-replacing amounts of income.
  - dx034 10y ago
    I assume from these numbers that you only have a few GB of traffic per month. If your page ever became popular this would quickly change.
    - brianwawok 10y ago
      Or he has a very skinny page :) Not a 3 MB js monstrosity.
thijsvandien 10y ago
With S3 Standard essentially getting S3 Standard - Infrequent Access storage pricing, where does that leave the latter?
- larrymcp 10y ago
  Hey, did you mean Reduced Redundancy (not Infrequent Access)? I just noticed that the new pricing for Standard (2.3¢) is now less than the pricing for Reduced Redundancy (2.4¢)! So there appears to be no reason to use Reduced Redundancy anymore.
  - thijsvandien 10y ago
    You're right, I mixed up after spending too much time on the different pricing pages yesterday. :)
woah 10y ago
What is the mechanism that makes it cheaper to take longer getting data out? Is it that they save money on a lower-throughput interface to the storage? Is it simply just market segmentation?
- RhodesianHunter 10y ago
  Tape is cheaper than hard drives are cheaper than flash drives are cheaper than ram.
- singlow 10y ago
  Storing the data in tape systems or some other system which is automated but not available in a matter of milliseconds.
- wmf 10y ago
  In theory, tape [1], optical [2], or spun-down disk [3] are cheaper but slower than spinning disk. Erasure coding [4] is also cheaper but slower than replication. One could even imagine putting cold data on the outer tracks of hard disks and warm on the inner tracks. In practice I suspect Glacier is market segmentation. [1] http://highscalability.com/blog/2014/2/3/how-google-backs-up-the-internet-along-with-exabytes-of-othe.html http://highscalability.com/blog/2014/2/3/how-google-backs-up... [2] http://www.everspan.com/home http://www.everspan.com/home [3] https://www.microsoft.com/en-us/research/publication/pelican-a-building-block-for-exascale-cold-data-storage/ https://www.microsoft.com/en-us/research/publication/pelican... [4] https://code.facebook.com/posts/1433093613662262/-under-the-hood-facebook-s-cold-storage-system-/ https://code.facebook.com/posts/1433093613662262/-under-the-...
  - ojiikun 10y ago
    it has more to do with clever uses of erasure coding than you might think.
- wallacoloo 10y ago
  Pure speculation here, so I'm probably completely wrong, but I imagine it's so that they can essentially reduce the amount of seeking. Glacier uses magnetic tape storage, I think, and I believe each tape has to physically be inserted into a machine to be read, and then be removed afterward. So there would be some downtime as tapes are swapped in/out. Therefore, it would make sense to aggregate reads ahead of time and maybe even physically reorder whole tape accesses to reduce the time it takes to load them. But this wouldn't explain why the read rate factors into cost. Maybe they scatter data across tapes as well, and higher bandwidth requires loading more tapes concurrently? Again, total conjecture. Please let me know which parts are wrong and which are right.
questionr 10y ago
how does this compare to Google's Coldline storage?
- vgt 10y ago
  Google has a "Glacier vs Nearline" calculator, which doesn't appear to have been updated yet: https://cloud.google.com/pricing/tco/storage-nearline https://cloud.google.com/pricing/tco/storage-nearline (work at Google)
- boulos 10y ago
  The biggest difference is that Glacier is still a "suspend/resume" type of accesss. However, if you just want to compare pricin, it'll depend on your access pattern and object sizes. Retrieval in all Google Cloud Storage is instant and for Coldline is $.05/GB (and Nearline $.01/GB). If you value that instant access, it seems the closest you'd get with the updates to Glacier is via the Expedited retrieval ($.03/GB and $.01/"request" which is per "Archive" in Glacier). Then you have to decide how much throughput you want to guarantee at $100/month for each 150MB/s. (It's naturally unclear since it was just announced what kind of best-effort throughput we're talking about without the provisioned capacity). If you're never going to touch the bytes, and each Archive is big enough to make the 40 KB of metadata negligible then the new $.004/GB/month is a nice win over Coldline's $.007. Somewhere in between and one of the bulk/batch retrieval methods might be a better fit for you. But again, it's still a bit of a challenge to go in and out of Glacier while Coldline (and Nearline and Standard) storage in GCS is a single, uniform API. That's worth a lot to me, and our customers. But if Glacier were a good fit for a problem you have, and you're talking about enough money to make the pain worth it, you should seriously consider it. Disclosure: I work on Google Cloud, so naturally I'd want you to use GCS ;).
deafcalculus 10y ago
Any chance Google will match this price for their coldline storage? I was planning to archive a few TBs in Google coldline, but Glacier is now cheaper and has a sane retrieval pricing model.
lucb1e 10y ago
If costs matter to you, e.g. for home backups, don't buy Glacier (and heck don't buy S3). A 3TB drive costs about 110eur, so if you'd have to buy a new one every year (you don't) that'd cost 110/3/1000/12=0.31 cents per gigabyte per month. Glacier? 7 times more expensive at 2.3ct. Hardware is usually not a business' main cost but it does matter for home users, small businesses or startups that didn't get funded yet, some of whom might consider Tarsnap or some other online storage solution which uses Glacier at best and S3 at worst. Now you could suddenly be 7× cheaper off if you do upkeep yourself (read: buy a raspberry pi) and if you throw away drives after one year.
- x0x0 10y ago
  There is value to having off-premises replicated storage on something more durable than home-user targeted drives. Google cloud nearline costs $0.12 per gigabyte-year with prices that will continue to fall. For a typical 500g hard drive that saw perhaps 700g of unique data, that's $84/year to have an outside-the-house replicated backup using something like Arq.
- hvidgaard 10y ago
  They are not the same thing at all. Glacier is the backup of the backup. It's where you go if your house burns down and the offsite backup at a relatives house is destroyed as well. If you want to compare them, you have to buy space on a different continent, and store your backup there.
- hga 10y ago
  I had such a setup when the Joplin 2011 tornado hit: http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/ http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/ and I got off lightly. But the separate room my BackupPC hard drives were in was breached (see 302-2nd-bathroom-with-hole-of-unknown-origin) and those drives become fit only for Ontrack's $$$ recovery service, maybe, and one of my computers with e.g. my email was seriously damaged. The data on it was easily recovered from rsync.net's off site Denver location, who's service I love and will continue to use for my most important and "hot" data. LTO (-4) tape had gotten capacious and cheap enough that I went back to tape (I'd outgrown DAT); if I didn't have a big sunk cost in a well working tape system and pool of tapes, which are very easy to put in e.g. a safe deposit box (they're a bit fragile, but nothing like a hard drive), I'd already be using one of S3, Glacier, or Backblaze, maybe even GCS since suddenly and irretrievably losing access to my backup data because a bot decided I was evil would not likely coincide with a total data loss at home (Google simply cannot be trusted if you're small fry like myself, as HN has been discussing as of late). As Glacier has gotten sane enough to use without twisting your mind into a pretzel, with the new price reduction for slow retrieval I can seriously think about adding it to the mix and switching to it when my LTO-4 tape drive dies someday (e.g. ~3TiB for ~$12/month per my quick calculation just now), instead of buying another tape drive.
jakozaur 10y ago
Great discount. I'm only surprised that Infrequent Access doesn't get any discount. By the way, I wrote article, how to reduce S3 costs: https://www.sumologic.com/aws/s3/s3-cost-optimization/ https://www.sumologic.com/aws/s3/s3-cost-optimization/
codedeadlock 10y ago
Has anyone tried to migrate to Backblaze. Their pricing seems really aggressive but I am not sure if we can compare Amazon and Backblaze when it comes to reliability. https://www.backblaze.com/b2/cloud-storage-providers.html https://www.backblaze.com/b2/cloud-storage-providers.html
- boulos 10y ago
  I love the folks at backblaze but the single datacenter thing really worries me (and again, disclosure, I work on Google Cloud). If you're just using it as another backup, maybe that's less of a concern: your house would have to burn down at the same time that they have a catastrophic failure. But it is part of the reason you see S3 and GCS costing more (plus the massive amount of I/O you can do to either S3 or GCS; I'd be curious what happens when there's a run on the Backblaze DC). Again, huge disclosure: I work on Google Cloud.
  - jorangreef 10y ago
    "But it is part of the reason you see S3 and GCS costing more" I bet that when Backblaze increase their scale by adding data centers they will decrease prices, not increase them. According to Ford's laws of service: Price, volume (scale) and quality are never opposed. 1. Decrease price and you can increase volume. 2. Increase volume and you can increase quality. 3. Increase quality and you can increase volume. 4. Increase volume and you can decrease price.
    - boulos 10y ago
      Sorry if I wasn't clear: your bytes on GCS and S3 are stored across multiple buildings (GCS Regional, S3 Standard). More copies is more dollars not less ;).
      - jorangreef 10y ago
        "More copies is more dollars not less ;)" As far as I am aware GCS does erasure coding across sites? Backblaze could do multiple tiers of erasure coding and they would still be able to reduce prices given more scale, ceteris paribus. It's not a question of number of replicas, data centers or technical implementation, but a question of pricing policy. Does one want to use volume and scale to drive prices down (and cheaper prices to increase volume) or does one want to use volume and scale to bloat margins? Backblaze are arguably doing the former. Does one want to lock customers into an ecosystem by enforcing excessive bandwidth prices or does one want to pass on bandwidth cost-savings to customers? Backblaze are arguably doing the latter. Backblaze would continue to be cheaper because their pricing policy serves customers across all dimensions. More scale is definitely less dollars not more (even if it means a fraction of a few more erasure coded shards across sites). Disclosure: I do not work for Backblaze.
adorable 10y ago
Any EBS storage price reductions? Those are pretty high at this stage.
MrBuddyCasino 10y ago
Do we know now how Glacier actually works? Tape robots, spun-down disks, racks of optical media? Best source I could find was: https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...
nakodari 10y ago
In our startup, the biggest cost is bandwidth. We live in an age where videos can be created and streamed in seconds to millions of people. With so high cost for bandwidth, it's very difficult for bootstrapped startups to grow as quickly as those who raise VC funding. I hope AWS can reduce the outgoing bandwidth cost by 50%.

293 points by jeffbarr 10y ago

QUFB 10y ago

I currently use S3 Infrequent Access buckets for some personal projects. These Glacier price reductions, along with the much better retrieval model look really great. However using Glacier as a simple store from the command-line seems horribly convoluted: https://docs.aws.amazon.com/cli/latest/userguide/cli-using-glacier.html https://docs.aws.amazon.com/cli/latest/userguide/cli-using-g... Does anyone know of any good tooling around Glacier for the command line?

jinl 10y ago
Perhaps you can script in python using boto3.
res0nat0r 10y ago
If you don't want to mess with all of that, you could use the standard "aws s3" command to upload your files to your s3 bucket like normal, and just apply an archive policy to your bucket or archive/ prefix or whatnot and it will automatically transfer your files to glacier for you.
- phonon 10y ago
  Yup, works great.

Alex3917 10y ago

While I'm not going to complain about a price reduction, I'd honestly be more excited if S3 implemented support for additional headers and redirect rules. Right now, anyone hosting a single page app (e.g. Angular/React) behind S3 and Cloudfront is going to get an F on securityheaders.io. And even worse, there is no way to prerender an SPA site for search engines without standing up an nginx proxy on ec2, which completely eliminates almost all of the benefits from Cloudfront. This is because right now S3 can only redirect based on a key prefix or error code, not based on a user agent like Googlebot or whatever. This means that even if you can technically drop a <meta name="fragment" content="!"> tag in your front end and then have S3 redirect on the key prefix '?_escaped_fragment_=', that will be a 301 redirect. This means that Google will ignore any <link rel="canonical" href="..."> tag on the prerendered page and will instead index https://api.yoursite.com https://api.yoursite.com or wherever your prerendered content is being hosted rather than your actual site. Not only is it a bunch of extra work to stand up an nginx proxy as a workaround, but it's also a whole extra set of security concerns, scaling concerns, etc. Not a good situation. edit: For more info on the prerendering issues, c.f.: https://github.com/prerender/prerender/issues/93 https://github.com/prerender/prerender/issues/93 https://gist.github.com/thoop/8165802 https://gist.github.com/thoop/8165802

ponyfleisch 10y ago
If i'm not wrong, cloudfront actually follows 302 redirects internally instead of passing the redirect on to the client. but since ?_escaped_fragment_= is a suffix, not a prefix, i don't think redirection rules help.
- Alex3917 10y ago
  Confusingly, _escaped_fragment_= is always a prefix if you have a #! in your url, otherwise I think it can come anywhere in the URL params.[1] Here is the exact system of redirects we were using, along with an explanation of why it didn't work: https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving-prerendered-s3-sites https://www.fwdeveryone.com/t/mdDouBesQwCv0za_o7GjCA/serving... [1] https://prerender.io/js-seo/angularjs-seo-get-your-site-indexed-and-to-the-top-of-the-search-results/ https://prerender.io/js-seo/angularjs-seo-get-your-site-inde...
  - ponyfleisch 10y ago
    Ah, yes, if you use #!. In that case, can't you do a cloudfront custom behaviour on /?_escaped_fragment_=*? I havent't tried this.
tbrock 10y ago
Hmm, as someone who is about to move from using a webserver to hosting our single page app on S3 you may have just convinced me not to. What exactly are the benefits over simply setting up nginx if not simplicity? Yeah, it's great to just serve the asset from s3 but the complexity of what you just described negates it almost entirely.
- Alex3917 10y ago
  > What exactly are the benefits over simply setting up nginx if not simplicity? We weren't fully aware of these limitations when we decided to host our site on S3. Had we been, we may have just used nginx. There are obviously a ton of benefits to S3 and Cloudfront, it's just than in practice you can't really get them if you need Google to index your site. And while Google claims they can now execute javascript and include async content, in practice this isn't true for any real Angular or React site. And even if every search engine were to magically execute js correctly, you'd still need to prerender your site in order for Facebook and Twitter to populate the preview cards for your site with the proper headline, summary, and image.

cpkpad 10y ago

Well, the costs are nicer, but mostly, Glacier goes from an unusable pricing model to a usable one. I was terrified to use Glacier. The previous model, if you made requests too rapidly, you might be hit with thousands of dollars of bills for relatively small data retrievals -- very easy to make a very expensive bug. I had wanted Amazon to wrap it in something where they managed that complexity for a long time. Looks like they finally did. Now the only thing Amazon needs to do is expand free tiers on all of their services, or at least very low cost ones. I prototype a lot of things from home for work -- kinda 20% time style projects where I couldn't really budget resources for it. The free tier is great for that. All services ought to have it -- especially RDS. I ought to be able to have a slice of a database (even kilobytes/tens of accesses/not-guaranteed security/shared server) paying nothing or pennies.

Johnny555 10y ago
A t2.large costs 10 cents/hour, a t2.medium RDS instance costs 7 cents/hour. If you put in 50 hours/month on this side project, that's $8.50 for compute plus maybe $3 for ~ 30GB of storage. $11.50/month doesn't sound too hard to budget for.
- andrioni 10y ago
  It might be more about bureaucracy than about cost. At my last job even small expenses would require printing forms and getting the CFO to sign them, so in the end it was pretty much not worth it for small tests.
  - brianwawok 10y ago
    Your work can't give you a few VMs in a lab somewhere that you can get free rein to prototype on? That is usually not too hard to get.. it seems the idea of the Amazon free tier is to give people a taste of AWS so they can decide to go in more or not. It's not really designed to be a free prototype for existing large customers product. Like the other poster said, you can host a tiny VM for $6 or so a month, not a big expense. If you are asking for $4000 a month production cluster, yes that is harder to just get..

scrollaway 10y ago

Anyone else finding their S3 bill consisting of mostly PUT/COPY/POST/LIST queries? Our service has a ton of data going in, very little going out and we're sitting with 95% of the bill being P/C/P/L queries and only the remaining 5% being storage. Either way, good news on the storage price reductions :)

cyberferret 10y ago
What app/site are you using to upload to S3? I use a combination of CloudBerry Backup and Arq Backup on my Macs/PCs here and the requests aren't that high (on average about 30Gb of data per machine in around 300K files). I am guessing it comes down to the algorithm used to compare and upload/download files. I believe the two solutions above use a separate 'index' file on S3 to track file compares.
- scrollaway 10y ago
  It's more that we have a pretty high throughput system, using Lambda. Users authenticate with an API gateway endpoint, we do a PUT to store a descriptor file, send a presigned PUT URL back so they can upload their file, we then process the file and do a COPY+DELETE to move it out of the "not yet processed" stage and finally do another PUT to upload the resulting processed file. Despite a lot of data, the storage bill is barely scratching $40, but we're at almost $700/mo on API calls.
  - cyberferret 10y ago
    Ah, thanks for the extra info. We have several web apps that take user uploads and store it to S3 buckets here too, but still we don't see an adversely high request load. Not sure if the handshaking involved in getting a pre-signed URL will be upping your count? We just use the AWS SDK on our Ruby back end. The user file is first uploaded to the (EC2) app server, then we use the SDK call to transfer it to the S3 bucket. Our storage and request costs are about equal at this stage. Using Lambda/Node, I guess that the SDK is not an option and you have to use the pre-signed URL method? Or else use Python and the SDK library?

sfdksfjkasfas 10y ago

43434

user5994461 10y ago

That's both a good and a terrible change. - The price reduction on S3 is good! Kudos AWS. - The price change on glacier is a fucking disaster. They replaced the _single_ expensive glacier fee with the choice among 3 user selectable fee models (Standard, Expedited, Bulk). It's an absolute nightmare added on top of the current nightmare (e.g. try to understand the disks specifications & pricing. It takes months of learning). I cannot follow the changes, too complicated. I cannot train my devs to understand glacier either, too much of a mess. AWS if you read this: Please make your offers and your pricing simpler, NEVER more complicated. (Even a single pricing option would be significantly better than that, even if its more expensive.)

tw04 10y ago
Simple is easy to shop around. Their goal is not to make pricing simple, don't hold your breath.
Lazare 10y ago
I believe this HUGELY simplifies glacier pricing. I don't think you understand just how insanely, incredibly, bizarrely complicated the old glacier pricing was.
- user5994461 10y ago
  There are 3 pricing models for S3, the 3rd one (glacier) having 3 sub-pricing models to be chosen at time of requests. I don't think you realize how insanely complex the entire S3 pricing model once you get out of the "standard price". Maybe I just have too much empathy for my poor devs and ops who try to understand how much what they're doing is gonna cost them. It's only one full page, both sided, of text after all.
  - Lazare 10y ago
    I'm not saying that S3 isn't complicated, or that the pricing model, even after the changes, isn't nuts. I'm saying that anyone who thinks this is more complicated than it was does not understand just how crazy glacier pricing was before. Three static glacier pricing tiers is a lot better than the previously system which is so complex that earlier versions of the AWS pricing page just gave up and called it "free". (Briefly: The old Glacier model's pricing wasn't based around data transfer, but on your automatically retroactively provisioned data transfer capacity based on your peak data transfer rate, billed on a sliding scale as if you'd maintained that rate for the entire month. There's probably a less intuitive way to bill people for downloading data, but if so I've never seen it. It was a system seemingly designed to prevent users from knowing what any given retrieval would cost.)

physcab 10y ago

This a really dumb question, but since I've never used Glacier what does the workflow for a Glacier application look like? I'm used to the world of immediate access needs, and fast API responses, so I can't imagine sending off a request to an API with a response "Your data will be ready in 1-5 hours, come back later".

needcaffeine 10y ago
It may be for backing up infrequently accessed data (compliance logs, etc) for example. Hypothetical: you create a logging service for users to send all their log data to you. You promise 365 days of archives, but 30 days of data accessible at any time. You create a lifecycle rule on your S3 bucket to automatically archive data to Glacier 30 days after creation. On the 31st day, your user decides they want to look at an old log. They click the big Download button. You display a message saying they'll get an email from you when that data is ready to download.
scrollaway 10y ago
Glacier is not for data you want readily available. It's for when you care more about storage than access.
- pauloday 10y ago
  Right, but if you want to retrieve some data through their API, how does it work? Normally you open the connection, ask for the data, then receive it and close the connection, does that change if there's a 5+ hour wait between the ask and the receive? Do you just leave the connection open? Provide them with a webhook to call when it's ready? I don't personally care about the answer but I'm pretty sure that's what they were asking.
  - extra88 10y ago
    At work our needs are simple, we manually run the aws cli to sync files up to S3 where there's a 1 day lifecycle policy to move them to Glacier. We don't use the API for restores, we do those through the web console and check back in a few hours to see if the files are downloadable. I think through the API you do not leave the connection open, you check with whatever frequency you want and when it's ready, the response will include the temporary location on S3 for the file.

jaytaylor 10y ago

EDIT: My mistake, this is the new S3 pricing! NOT Glacier pricing! Thank you res0nat0r. Am I understanding this right? $0.023/GB/month for Glacier, so * 12 months/year = $0.276/GB/year, which means: 10GB = $2.70/year 100GB = $27.00/year 1TB = $270.00/year ... And this is only the storage cost. This doesn't take into account the cost should you actually decise to retrieve the data. So considering a 1TB hard drive [0] costs $50.00, how is this cost effective? I can buy 5x1TB hard drives for the price of 1TB on Glacier. I understand there is overhead to managing it yourself. So, is this just not targeted to technically proficient folks? [0] https://www.amazon.com/Blue-Cache-Desktop-Drive-WD10EZEX/dp/B0088PUEPK https://www.amazon.com/Blue-Cache-Desktop-Drive-WD10EZEX/dp/...

acdha 10y ago
S3 data is backed up in three geographically separated data-centers, so you need to budget for 3 copies. It's a running service (rather than cold storage) which means you need servers, power network ports, etc. Again, times 3. Finally, include the software development to build the server stack and the staff for 24x7 ops and security. It's possible to beat their pricing but most people who actually do manage it by cutting in an area which they don't need.
eli 10y ago
Where are you going to keep it? On site? Not so good for disaster recovery.
- deleted 10y ago
  [deleted]
derekdahmer 10y ago
That's comparing apples and oranges - your hard drive doesn't live in a secure, fireproof data center in the cloud you can access from anywhere.
- jeffbarr 10y ago
  Or eggs and omelettes...

Perceptes 10y ago

Is anyone using either S3 or Glacier to store encrypted backups of their personal computer(s)? I've only used Time Machine to back up my machine for a long time, but I don't really trust it and would like to have another back up in the cloud. Any tools that automate back up and restore to/from S3/Glacier? What are your experiences?

res0nat0r 10y ago
Check out Duplicity or Duplicati. Also Arq for OSX I believe is very nice and has a great GUI but isn't free.
- jonatron 10y ago
  +1 for Duplicity
icecreammatt 10y ago
I use Arq (https://www.arqbackup.com/ https://www.arqbackup.com/) and it works very well. I've only tested retrieving small amounts of data from it so I can't comment much on a large bill. I only wish it worked on Linux. I've been thinking about seeing if it would work with Wine.
- chrislund 10y ago
  Ditto. I've been using Arq for 4 years now to back up nearly a TB of data to S3 and Google Cloud Storage.
  - CoachRufus87 10y ago
    Out of curiosity, why both?
    - chrislund 10y ago
      I guess mostly to not have all my cloud backup eggs in one basket.

msravi 10y ago

> For example, retrieving 500 archives that are 1 GB each would cost 500GB x $0.01 + 500 x $0.05/1,000 = $5.25 Shouldn't that be $5.025? Or did I misunderstand?

la6470 10y ago

This is just awesome ... AWS continue to surprise with new features.. now only if I could mark a bucket encrypted so anything uploaded automatically gets encrypted!!

ww520 10y ago

Is the outgoing bandwidth still the same price? Bandwidth cost is kind of high compared to other services.

dx034 10y ago
Yes, they are high for all 3 major cloud providers. I guess it's to keep you from using the cheapest part of each provider. That way you have to use their whole ecosystem and they can offer some products below costs to lure people in.

DanBlake 10y ago

Would really like to see some massive reductions in the operation costs and most importantly, bandwidth costs. The bandwidth costs are so far out of line with what the network transfer actually costs, it just feels like price fixing between the major cloud players that nobody is drastically reducing those prices, only storage prices. Charging 5 cents per gigabyte (at their maximum published discount level) is the equivalent to paying $16,000 per month for a 1 gigabit line. This does not count any operation costs either, which could add thousands in cost as well, depending on how you are using S3. There are several providers that offer a unmetered 1gbps line PLUS a dedicated server for ~600-750/mo. Providers like OVH offer the bandwidth for as little as 100/month. ( https://www.ovh.com/us/dedicated-servers/bandwidth-upgrade.xml https://www.ovh.com/us/dedicated-servers/bandwidth-upgrade.x... ) I am just not sure how amazon can justify a 160x price increase over OVH or a 30x increase over dedicated server + transfer. For the time being, the best bet is to use S3 for your storage and then have a heavily caching non amazon CDN on top of it (like cloudflare) to save the ridiculous bandwidth costs.

Scoundreller 10y ago
I use s3 as a webserver for static pages. Total bandwidth costs: 17 cents a month. 14 or so cents now. And I have some tens of thousands of visits per month. The only cognitive thought I've had to put in since setup was questioning one aspect of a bill (it amounted to pennies, but I couldn't figure it out, and maybe they were overbilling bigger entities). And they gave me a month free (including DNS costs). I wish the Route53 would go down... Hard to beat, even if I were running job-replacing amounts of income.
- dx034 10y ago
  I assume from these numbers that you only have a few GB of traffic per month. If your page ever became popular this would quickly change.
  - brianwawok 10y ago
    Or he has a very skinny page :) Not a 3 MB js monstrosity.

thijsvandien 10y ago

With S3 Standard essentially getting S3 Standard - Infrequent Access storage pricing, where does that leave the latter?

larrymcp 10y ago
Hey, did you mean Reduced Redundancy (not Infrequent Access)? I just noticed that the new pricing for Standard (2.3¢) is now less than the pricing for Reduced Redundancy (2.4¢)! So there appears to be no reason to use Reduced Redundancy anymore.
- thijsvandien 10y ago
  You're right, I mixed up after spending too much time on the different pricing pages yesterday. :)

woah 10y ago

What is the mechanism that makes it cheaper to take longer getting data out? Is it that they save money on a lower-throughput interface to the storage? Is it simply just market segmentation?

RhodesianHunter 10y ago
Tape is cheaper than hard drives are cheaper than flash drives are cheaper than ram.
singlow 10y ago
Storing the data in tape systems or some other system which is automated but not available in a matter of milliseconds.
wmf 10y ago
In theory, tape [1], optical [2], or spun-down disk [3] are cheaper but slower than spinning disk. Erasure coding [4] is also cheaper but slower than replication. One could even imagine putting cold data on the outer tracks of hard disks and warm on the inner tracks. In practice I suspect Glacier is market segmentation. [1] http://highscalability.com/blog/2014/2/3/how-google-backs-up-the-internet-along-with-exabytes-of-othe.html http://highscalability.com/blog/2014/2/3/how-google-backs-up... [2] http://www.everspan.com/home http://www.everspan.com/home [3] https://www.microsoft.com/en-us/research/publication/pelican-a-building-block-for-exascale-cold-data-storage/ https://www.microsoft.com/en-us/research/publication/pelican... [4] https://code.facebook.com/posts/1433093613662262/-under-the-hood-facebook-s-cold-storage-system-/ https://code.facebook.com/posts/1433093613662262/-under-the-...
- ojiikun 10y ago
  it has more to do with clever uses of erasure coding than you might think.
wallacoloo 10y ago
Pure speculation here, so I'm probably completely wrong, but I imagine it's so that they can essentially reduce the amount of seeking. Glacier uses magnetic tape storage, I think, and I believe each tape has to physically be inserted into a machine to be read, and then be removed afterward. So there would be some downtime as tapes are swapped in/out. Therefore, it would make sense to aggregate reads ahead of time and maybe even physically reorder whole tape accesses to reduce the time it takes to load them. But this wouldn't explain why the read rate factors into cost. Maybe they scatter data across tapes as well, and higher bandwidth requires loading more tapes concurrently? Again, total conjecture. Please let me know which parts are wrong and which are right.

questionr 10y ago

how does this compare to Google's Coldline storage?

vgt 10y ago
Google has a "Glacier vs Nearline" calculator, which doesn't appear to have been updated yet: https://cloud.google.com/pricing/tco/storage-nearline https://cloud.google.com/pricing/tco/storage-nearline (work at Google)
boulos 10y ago
The biggest difference is that Glacier is still a "suspend/resume" type of accesss. However, if you just want to compare pricin, it'll depend on your access pattern and object sizes. Retrieval in all Google Cloud Storage is instant and for Coldline is $.05/GB (and Nearline $.01/GB). If you value that instant access, it seems the closest you'd get with the updates to Glacier is via the Expedited retrieval ($.03/GB and $.01/"request" which is per "Archive" in Glacier). Then you have to decide how much throughput you want to guarantee at $100/month for each 150MB/s. (It's naturally unclear since it was just announced what kind of best-effort throughput we're talking about without the provisioned capacity). If you're never going to touch the bytes, and each Archive is big enough to make the 40 KB of metadata negligible then the new $.004/GB/month is a nice win over Coldline's $.007. Somewhere in between and one of the bulk/batch retrieval methods might be a better fit for you. But again, it's still a bit of a challenge to go in and out of Glacier while Coldline (and Nearline and Standard) storage in GCS is a single, uniform API. That's worth a lot to me, and our customers. But if Glacier were a good fit for a problem you have, and you're talking about enough money to make the pain worth it, you should seriously consider it. Disclosure: I work on Google Cloud, so naturally I'd want you to use GCS ;).

deafcalculus 10y ago

Any chance Google will match this price for their coldline storage? I was planning to archive a few TBs in Google coldline, but Glacier is now cheaper and has a sane retrieval pricing model.

lucb1e 10y ago

If costs matter to you, e.g. for home backups, don't buy Glacier (and heck don't buy S3). A 3TB drive costs about 110eur, so if you'd have to buy a new one every year (you don't) that'd cost 110/3/1000/12=0.31 cents per gigabyte per month. Glacier? 7 times more expensive at 2.3ct. Hardware is usually not a business' main cost but it does matter for home users, small businesses or startups that didn't get funded yet, some of whom might consider Tarsnap or some other online storage solution which uses Glacier at best and S3 at worst. Now you could suddenly be 7× cheaper off if you do upkeep yourself (read: buy a raspberry pi) and if you throw away drives after one year.

x0x0 10y ago
There is value to having off-premises replicated storage on something more durable than home-user targeted drives. Google cloud nearline costs $0.12 per gigabyte-year with prices that will continue to fall. For a typical 500g hard drive that saw perhaps 700g of unique data, that's $84/year to have an outside-the-house replicated backup using something like Arq.
hvidgaard 10y ago
They are not the same thing at all. Glacier is the backup of the backup. It's where you go if your house burns down and the offsite backup at a relatives house is destroyed as well. If you want to compare them, you have to buy space on a different continent, and store your backup there.
hga 10y ago
I had such a setup when the Joplin 2011 tornado hit: http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/ http://www.ancell-ent.com/1715_Rex_Ave_127B_Joplin/images/ and I got off lightly. But the separate room my BackupPC hard drives were in was breached (see 302-2nd-bathroom-with-hole-of-unknown-origin) and those drives become fit only for Ontrack's $$$ recovery service, maybe, and one of my computers with e.g. my email was seriously damaged. The data on it was easily recovered from rsync.net's off site Denver location, who's service I love and will continue to use for my most important and "hot" data. LTO (-4) tape had gotten capacious and cheap enough that I went back to tape (I'd outgrown DAT); if I didn't have a big sunk cost in a well working tape system and pool of tapes, which are very easy to put in e.g. a safe deposit box (they're a bit fragile, but nothing like a hard drive), I'd already be using one of S3, Glacier, or Backblaze, maybe even GCS since suddenly and irretrievably losing access to my backup data because a bot decided I was evil would not likely coincide with a total data loss at home (Google simply cannot be trusted if you're small fry like myself, as HN has been discussing as of late). As Glacier has gotten sane enough to use without twisting your mind into a pretzel, with the new price reduction for slow retrieval I can seriously think about adding it to the mix and switching to it when my LTO-4 tape drive dies someday (e.g. ~3TiB for ~$12/month per my quick calculation just now), instead of buying another tape drive.

jakozaur 10y ago

Great discount. I'm only surprised that Infrequent Access doesn't get any discount. By the way, I wrote article, how to reduce S3 costs: https://www.sumologic.com/aws/s3/s3-cost-optimization/ https://www.sumologic.com/aws/s3/s3-cost-optimization/

codedeadlock 10y ago

Has anyone tried to migrate to Backblaze. Their pricing seems really aggressive but I am not sure if we can compare Amazon and Backblaze when it comes to reliability. https://www.backblaze.com/b2/cloud-storage-providers.html https://www.backblaze.com/b2/cloud-storage-providers.html

boulos 10y ago
I love the folks at backblaze but the single datacenter thing really worries me (and again, disclosure, I work on Google Cloud). If you're just using it as another backup, maybe that's less of a concern: your house would have to burn down at the same time that they have a catastrophic failure. But it is part of the reason you see S3 and GCS costing more (plus the massive amount of I/O you can do to either S3 or GCS; I'd be curious what happens when there's a run on the Backblaze DC). Again, huge disclosure: I work on Google Cloud.
- jorangreef 10y ago
  "But it is part of the reason you see S3 and GCS costing more" I bet that when Backblaze increase their scale by adding data centers they will decrease prices, not increase them. According to Ford's laws of service: Price, volume (scale) and quality are never opposed. 1. Decrease price and you can increase volume. 2. Increase volume and you can increase quality. 3. Increase quality and you can increase volume. 4. Increase volume and you can decrease price.
  - boulos 10y ago
    Sorry if I wasn't clear: your bytes on GCS and S3 are stored across multiple buildings (GCS Regional, S3 Standard). More copies is more dollars not less ;).
    - jorangreef 10y ago
      "More copies is more dollars not less ;)" As far as I am aware GCS does erasure coding across sites? Backblaze could do multiple tiers of erasure coding and they would still be able to reduce prices given more scale, ceteris paribus. It's not a question of number of replicas, data centers or technical implementation, but a question of pricing policy. Does one want to use volume and scale to drive prices down (and cheaper prices to increase volume) or does one want to use volume and scale to bloat margins? Backblaze are arguably doing the former. Does one want to lock customers into an ecosystem by enforcing excessive bandwidth prices or does one want to pass on bandwidth cost-savings to customers? Backblaze are arguably doing the latter. Backblaze would continue to be cheaper because their pricing policy serves customers across all dimensions. More scale is definitely less dollars not more (even if it means a fraction of a few more erasure coded shards across sites). Disclosure: I do not work for Backblaze.

adorable 10y ago

Any EBS storage price reductions? Those are pretty high at this stage.

MrBuddyCasino 10y ago

Do we know now how Glacier actually works? Tape robots, spun-down disks, racks of optical media? Best source I could find was: https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...

nakodari 10y ago

In our startup, the biggest cost is bandwidth. We live in an age where videos can be created and streamed in seconds to millions of people. With so high cost for bandwidth, it's very difficult for bootstrapped startups to grow as quickly as those who raise VC funding. I hope AWS can reduce the outgoing bandwidth cost by 50%.