15 ms·

S3 isn't getting cheaper

288 points by rckrd 4y ago

paulpauper 4y ago
all amazon services seem so expensive, hundreds of dollars/ month bills easily for running a few large ec2 servers all month.
- snoopy_telex 4y ago
  Why haven't there been a upstart taking market share? Why do only the big players get into the game and hold similar costs?
  - rckrd 4y ago
    Large fixed costs, economies of scale, whole product (need compute AND storage). It's possible that a company like Cloudflare can disrupt certain verticals in storage that are mispriced by AWS yet have a larger than expected TAM.
  - vorpalhex 4y ago
    DigitalOcean, Linode, Hetzner, Vultr...
    - jonatron 4y ago
      DO's spaces didn't even work for the most basic test of putting a few images on a page when I tried it. I couldn't say it's comparable to S3.
      - ipaddr 4y ago
        Something tells me you are doing something wrong.
      - vorpalhex 4y ago
        I use DO spaces heavily - for images no less. It works great. What issues did you have?
        jonatron 4y ago
        Uploaded about 100 images. Put those images on a page. 50% of them timed out when downloading. Maybe I was just unlucky, but it was a good way to make me instantly lose all confidence DO Spaces. S3 and Backblaze B2 worked fine for the same thing at the same time as when this happened.
        vorpalhex 4y ago
        That's definitely incorrect behavior. Did you reach out to support?
    - upupandup 4y ago
      Which raises my question: Is there some open source github project that lets you create an S3 API compatible with underlying heterogeneous VPS hosts clustered? I'm guessing even with this storage would still be expensive, so how does Backblaze pull this off?
      - vorpalhex 4y ago
        Yes, several. Minio is the big one but I believe Ceph can do this too. Others exist. Disk is cheap. Real cheap.
        upupandup 4y ago
        ah yes minio! the one issue i have with using VPS for production is, are they hardened? or is it enough just with proper unix user management and UFW? I have this fear that the VPS box has attack surfaces or zero day vulnerabilities but when I am on the cloud I do not have this worry. Perhaps irrational but can't argue with the peace of mind that expensive clouds offer. Although we've seen misconfigured S3 buckets leaking data so.
        vorpalhex 4y ago
        What makes you believe the threat model for AWS is different than DO? They are just VMs at the end of the day.
  - waitTho 4y ago
    Human biology is not programmed to abide conflict of interest. On paper things can be written just so, but what’s on paper does not stop feelings, friendship, and connection between two people from forming given biology. Pointing at some philosophy to equivocate away physical laws is the simple con leveraged against the people. The big players leverage their understanding of science and well paid lawyers to play a cognitive game where investment in storage is set aside, as storage is “a solved problem”, they collude to focus government spend on new things they can charge consumers for after charging us via taxes and agency, to build it. Good luck finding a VC willing to compete against Bezos. They’re not going to target the guy managing the infra risk, providing a cheap platform key to their cheap startup gambling. They’re going to target naive college kids to try and build a rocket for them. Because VCs are smarter than Bezos; do none of the work, own the reward.
  - 0xbadcafebee 4y ago
    It's easy to enter the market and offer a cheap product. It's hard to enter the market and offer a very solid product. It's nearly impossible to enter the market and provide the hundreds of services the big players can provide, and operate at the scale they can provide, servicing the number of markets and customers they do, with the level of support they do. The big players know that what separates the majors from the minors is trust. If you buy from AWS, you know what you get works, and you will pay a premium for that assurance. And also it is really fricking expensive to be AWS.
    - jamesfinlayson 4y ago
      Very much this - I don't have much experience with providers in this space but Digital Ocean has taken maybe a decade to build and offer a small number of services. That said, there is a lot of AWS that I probably wouldn't know existed - I know in my day job I make big use of maybe five services, plus maybe another 10 glue services between them (CloudWatch, IAM, VPC etc).
  - epolanski 4y ago
    Because infrastructure is not that high of cost for most companies and every dev and devops out there knows AWS and has no clue how competitore dashboards even look like.
- Sebb767 4y ago
  AWS is great if you have lots of money and little time. Which was very true in the last ~decade, where we had lots of venture funded startups that had very much money, but little time. Also, while you definitely pay a markup, the standard EC2 pricing also contains instant availability. If you skip on buying a car and instead pay a taxi to wait 24/7, your costs will also be insanely high. Additionally, AWS provides a great ecosystem where your app can easily be managed in - things like getting a https certificate, setting up a redundant load balancer and even a CDN, database or a Kubernetes cluster can be made simply with a few clicks in an UI. If you don't have someone who knows how to configure those services, it can detract a lot from what you're actually trying to do as a business. Lastly, it has all these enterprise features you suddenly need - solid billing, encryption, certificates etc.. Don't get me wrong, it's expensive, but there's a reason so many businesses use AWS.
- chrisan 4y ago
  if all you are running are a few ec2 servers then you have quite a lot of competitors to choose from and price out/balance with reliability. we use aws at work, but been running side projects on digital ocean for years for way less than hundreds a month
bicijay 4y ago
Lately im doing Backblaze B2 + Bunny CDN and im very satisfied with the prices
- js4ever 4y ago
  Blackblaze reliability and performance are below aws, same for bunny cdn. Although I understand it can be interesting for some use cases where perf/reliability is not critical.
  - ape4 4y ago
    Counting today's outage? https://news.ycombinator.com/item?id=32267222 https://news.ycombinator.com/item?id=32267222
  - upupandup 4y ago
    I guess you get what you paid for but how much of a difference is there? The cost saving it advertises is quite alluring.
    - dilyevsky 4y ago
      Afaik Backblaze has maintenance every Thursday during which they don’t guarantee uptime.
  - coder543 4y ago
    Can you cite your sources? AWS "reliability" has been the direct cause of a number of sleepless nights for me over the years. Comparing to a few years ago when I worked on a large-scale product hosted on bare metal servers that worked beautifully, I don't think AWS is all it is hyped up to be. Anecdotal, I know, but even with no experience using Backblaze or Bunny, the bar they would have to meet is a lot lower than you're implying.
    - js4ever 4y ago
      I'm talking about my personal experience, on blackblaze the number of 500 errors was simply not acceptable for my use case, likewise for performance and latency. I was a bit disappointed by bunny cdn rps/latency. But indeed price is not at all comparable. Also I'm not talking about any aws service but more specifically about S3 a d CloudFront. Finally, as I said above those blackblaze and bunny are amazing if you try to optimize the cost as your main goal.
dtx1 4y ago
I suspect it might be an issue of "cheap enough". For smaller Projects, the s3 storage cost doesn't matter too much. Medium sized projects may find the tradeoff between price vs cost of a custom solution to still be more than good enough. Large enough projects simply chose a different, more specialized solution anyway
- jenny91 4y ago
  Big projects get custom discounts; there's a whole different pricing game for huge customers.
  - jopsen 4y ago
    Or they make lots of money and choose not to care.
    - daniel-cussen 4y ago
      There is such a thing. Like if it ain't broke don't fix it, compute costs aren't always that big, sometimes single-thread is enough even on Wall Street exchanges. And like come on originally the alternative was hiring employees and managers for those employees and dealing with human error and deception overlapping with the guilt of exploiting them, the whole management game. Difficult to find true leadership, and a work-ethic shared between manager and employee. And the education I got in the nineties and noughts was made for desk jobs with stationary, not computers. Sometimes automation alone is fine. Even on a computer that hasn't been reset in fifty years and is obsolete according to everyone else, hey if it does the trick. Make sure it stays powered. Cobol. Does the trick. Nuclear plants, they use really old software, one using very new software and connected to the internet was hit by the Morris Worm. I literally picked up a 32 ounce rock on the street that was intended, judging by its shape and way it was cut out of cement and pebble composite, for stoning. Like for Biblical harlots. It was left behind right after a protest was cleared one Friday afternoon on Portugal and Alameda, Santiago Centro, Santiago Chile. Found it Friday like in April, at 22:16. I roam, I checked out the scene--as is my wont--to see what's up, different graffiti on the walls, and then whoops don't see a lot of pint-sized rocks on the sidewalk, somebody might trip, better clear it. Took it home. Realized I should have worn gloves. Next time I went to hang out in front of the police station I told them about it, hey you could do forensics I said, they're like uh no wrong station for forensics, uh...thanks for clearing it away, those are meant for cops. I got stoned with similar rocks after watching cops on motorcycles retreat from a mob, didn't click that mob was throwing rocks, kind of aiming at whatever that moved, and fuck did I then move, I sprinted away to safety before the gates closed on me. Luckily I didn't take a direct hit, none nailed me. But the moral of the story is this: nothing about being tens or hundreds or thousands or in this case millions of years old renders it ineffective as a weapon. Same laws of physics. Same gravity. Muscle equal strength. Rock is just as hard now as it was then. Harder than my skull. Death is bad. Bad then, and bad now.
influx 4y ago
It looks like the most recent data point, S3 was actually cheaper than the raw storage, while providing 11 9's of durability, across multiple AZs. Still looks like a pretty good deal, as the price of storage has mostly flatlined in this graph. Nor does this even account for cold storage, or reduced redundancy.
- secabeen 4y ago
  The prices are in different units. S3 is $/GB-Mo, raw storage is in $/GB. So, when they are equal on the graph, you are paying each month to AWS what the cost of the raw disk is. Now, yes, you need a lot more than just raw disk to effectively store data, but even if you just assume a 5-year lifespan for that disk, the price difference displayed on the graph as equality is actually a 60x difference in plain $. The price has also appeared to flat-line because they are using a linear graph scale for logarithmic data. It should use a log-scale for the y-axis.
  - ceeplusplus 4y ago
    Now factor in redundancy and server costs, and the difference is not so huge. You're still paying a multiple of the raw cost, but unless you're storing on the exabyte scale I think it shouldn't really matter in the grand scheme of things. Cost of a single engineer to manage a Minio cluster probably already outweighs the extra cost you're paying at any reasonable scale (i.e. most companies). And if you're a big player the published costs are not what you're paying.
    - hedora 4y ago
      They use wide erasure code stripes, so redundancy is ~ 1x within a data center. Let's assume 3 data centers. It's well known how to build a storage node whose cost is mostly disks, let's say 50% of the hardware is not disks. So, redundancy and server costs explain up to a 6x mark up. Power and networking really shouldn't account for the other 54x.
      - jenny91 4y ago
Dunedan 4y ago
> Yet, AWS S3 pricing hasn't decreased as fast as the underlying storage costs. > […] > Another blog post analyzes the same theory for compute and finds a similar story using pricing data from AWS EC2. AWS used to do frequent price reductions years ago. At a certain point they seem to have stopped doing that and are now only doing them rarely. That's really a shame as there are still a lot of AWS offerings which are priced way too high (data transfer being the most prominent one). It'll be interesting to see if and up to which point AWS will keep the prices stable with raising inflation.
- aabhay 4y ago
  One of the benefits of inflation!
epberry 4y ago
The R2 vs S3 story is pretty interesting. For archival use cases, S3 still wins by a mile, but for running apps, R2 often wins (minus lacking features). I wrote up my findings here: https://www.vantage.sh/blog/cloudflare-r2-aws-s3-comparison https://www.vantage.sh/blog/cloudflare-r2-aws-s3-comparison
Dunedan 4y ago
> It's hard to come to a firm conclusion since pricing data is held very secretly (AWS quickly deletes announcements of historical price decreases). That's not correct. Here is the history of every (properly tagged) price reduction AWS ever announced: https://aws.amazon.com/blogs/aws/category/price-reduction/ https://aws.amazon.com/blogs/aws/category/price-reduction/
- rckrd 4y ago
  Those blog posts are a good start. I was referencing the full pricing page for s3 that breaks down by region and class. AFAIK there are some third parties that track the granular data, but it's not preserved anywhere on the AWS page. A lot of price decreases also might have happened silently (without a blog post) or as de facto decreases (widely but privately negotiated, e.g. "sticker price").
  - Dunedan 4y ago
    > I was referencing the full pricing page for s3 that breaks down by region and class. You were referring to the announcements and these announcements are still available. The actual pricing pages of course only show current prices. What would be the motivation for AWS to show outdated prices there as well? No other company I can think of does that and given the complexity of the pricing structure for AWS services that'd only confuse users even more. > A lot of price decreases also might have happened silently (without a blog post) Do you have an example for that or is this just hearsay? > or as de facto decreases (widely but privately negotiated, e.g. "sticker price") Privately negotiated deals aren't regular price reductions as they're only available for customers with a fairly large spend for such AWS services.
    - lstamour 4y ago
      I can say that price _increases_ happened for a few AWS products in specific regions/countries without blog posts. Often AWS simply emails affected customers 1-3 months in advance and after making the change. That said, I imagine prices shifted due to currency, labour costs, energy costs or other regional concerns. And I’ve seen the opposite - I’ve seen AWS freeze prices and eat the cost differences long after they should have raised costs. It varies by product and region and marketing strategy, I expect. The only rule of thumb is what is posted on the pricing page is supposed to be what you get charged. No APIs exist for the most part, and the actual charges can sometimes differ anyway (e.g. grandfathered pricing etc.)
Comevius 4y ago
As a European I take not getting many times as expensive, like our fuel, energy and food prices. I wonder how the European cloud will fare this winter, and how many European customers we will lose.
fxtentacle 4y ago
The main issue we have with S3 is the extortionary egress bandwidth fee. Storage pricing seems OK, but what's the point if I can't send those files to users?
- rckrd 4y ago
  A bit tangential, but in some cases you can use S3 Gateway endpoints, which can dramatically reduce your egress costs if you're just accessing it within a VPC. [0] https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpo...
  - judge2020 4y ago
    This is the entire point of the egress fees. If you can keep traffic inside AWS it's basically free. Getting it out costs money.
- tpmx 4y ago
  I think this would be an excellent area for regulation since it anti-competitive (portability is dramatically reduced since it has been made artifically expensive to move the data to another cloud service) and the cloud services haven't shown any interest in doing something about it on their own.
  - sfe22 4y ago
    Are you seriously suggesting that S3 is a monopoly with means to block competition. I mean even when discussing cloud in general, what you are suggesting is a joke.
    - tpmx 4y ago
      Anti-competitive practices (https://en.wikipedia.org/wiki/Anti-competitive_practices https://en.wikipedia.org/wiki/Anti-competitive_practices) don't require a monopoly.
  - ipaddr 4y ago
tick_tock_tick 4y ago
Cloudflare's entry into this market will be interesting no bandwidth costs might put real pressure on AWS to get reasonable with it's pricing. https://blog.cloudflare.com/introducing-r2-object-storage/ https://blog.cloudflare.com/introducing-r2-object-storage/
- epolanski 4y ago
  That sounds a lot like "let's hope the new AMD GPUs will be priced low so Nvidia will cut their prices"
  - tick_tock_tick 4y ago
    I mean they announced pricing already and it's much cheaper.... And are you implying that if AMD prices GPUs cheaply doesn't impact Nvidia?
tnolet 4y ago
For my company, neither egress nor storage cost are the big issue. It’s the API call (PUT) cost. We deal with payloads that are just a little too big for a database (we run Postgres and Clickhouse) but just too frequent (~100 per second) and small (think largish json blobs) to be effective on S3. We are write heavy. Reads are probably 1% but need to be instant for a good UI and API experience.
- YawningAngel 4y ago
  This is a very naive question so I might be very wrong, but isn't postgres pretty flexible about objects now?
  - tnolet 4y ago
    Yes, but it does blow up TOAST and has a lot of impact on the deletion behavior on busy tables. We removed all larger json blobs from PG. typical settings or config stuff in json in PG are fine. We use that all the time. But larger json blobs of several kilobytes are still an issue for semi timeseries data.
    - iknownothow 4y ago
      Could you elaborate on the TOAST issues you're having? We're pretty liberal with our use of large JSONB objects and might hit a billion objects in a year or so.
    - LunaSea 4y ago
      Have you tried using PostgreSQL Large Objects (LOs)?
  - michael1999 4y ago
    100/s is 3 billion per year. Postgres has a hard limit of 4 billion blobs in a column. that ignores the reality that RDS storage is comparatively expensive.
    - 4y ago
klodolph 4y ago
It looks like storage costs haven’t changed much since the last S3 price reduction five years ago. My other take on this is that given how slowly HDD costs are going down at this point, tape is going to remain relevant for some consumers for a lot longer than many of us thought.
Sebb767 4y ago
> Even with a slowdown of Moore's Law, it seems like AWS has a healthy margin to continue to offer strategic price cuts only when necessary. Which doesn't surprise me much, really. If your customers are mostly stuck with you, competition is sparse and people pay the price you demand - why would you reduce prices?
advisedwang 4y ago
Hard disks have been getting larger capacity but i/o capability is fairly flat. The result is that price/GB is dropping but price/iops is not. So _cold_ storage is where you would expect to see pricing fall. I don't follows AWS pricing closely but I've seen a lot of news around Glacier over the year so that might reflect this fact.
- aledalgrande 4y ago
  There's now a new Glacier product, called Glacier Instant Retrieval, which supposedly makes it cheaper and faster to retrieve from Glacier, if you do it a limited number of times.
  - pwarner 4y ago
    Yeah this is magic price wise
- didgetmaster 4y ago
  The capacity and speed curves of drives is quite different. When the first TB drives were released about 15 years ago, the fastest ones were about 100MB/s. Now you can get 20TB drives (20x capacity) and the fastest ones are about 300MB/s (3x speed). It is just far easier and cheaper to make a drive twice as big than it is to make it twice as fast. SSDs are a different animal, but have some similar characteristics. Within the same generation (e.g. m.2 pcie gen 4), you can get drives that have a lot more capacity but have roughly the same access speeds (i.e. the 2TB version is very similar to the 1TB version). The speed increases between generations is much better than with HDD. Pcie gen 3 drives seemed to max out about 3500MB/s while the gen 4 drives are about double that. I have seen reports that gen 5 drives might double it again With HAMR technology we might get HDD drives with capacities in the 50TB-100TB range. You can bet that the speed won't be 5x current technology even if they get dual-actuators in them. There will need to be some kind of breakthrough technology to improve it significantly. This is why we need better data management systems. If the meta-data (e.g. file table) is only 1% of the data that is still a lot of data to read in and store in RAM. We need better systems where the file records are much smaller. https://didgets.substack.com/p/where-did-i-put-that-file https://didgets.substack.com/p/where-did-i-put-that-file
kbumsik 4y ago
Well, at least AWS has not increased the price like GCP did: https://cloud.google.com/storage/pricing-announce https://cloud.google.com/storage/pricing-announce
coding123 4y ago
Because of inflation, the price is going down. However also because of inflation, I have had a 10% "pay cut" since mid last year.
mathgladiator 4y ago
Has anyone had any direct experience between S3 and Backblaze for the basic usecase of uploading and downloading files? https://www.backblaze.com/b2/cloud-storage-pricing.html https://www.backblaze.com/b2/cloud-storage-pricing.html Is just radically different.
- dmw_ng 4y ago
  B2 isn't great for low latency serving, objects that aren't in hot cache have extremely variable delays on first fetch, and the delay (at least as of a year ago) scales according to object size. For largeish video (over 500mb) I remember seeing >1 second latency, enough to rule out using it for anything public facing
aejnsn 4y ago
> AWS has a healthy margin to continue to offer _strategic price cuts_ only when necessary I’ve never seen a non-strategic price cut. :)
- smilekzs 4y ago
  I think the contrast is between: A: Cutting prices whenever costs go down B: Cutting prices only when competitors do so
  - deleted 4y ago
    [deleted]
- fomine3 4y ago
  Inflation, virtually!
fleddr 4y ago
Storage price reductions slowing down or even coming to a halt makes me wonder what this means for "infinite storage" companies. I'm talking services like Facebook, Youtube, that are "free" (ad supported) where on a daily basis an absurd amount of new content is added, yet almost nothing ever removed. If storage needs grow endlessly yet storage costs stopped going down, wouldn't that mean that the model in the long term is unsustainable? Sure you can delay the inevitable (compress content, move old stuff to cold storage) but ultimately storage costs per user goes up whilst income likely does not.
- prepend 4y ago
  If storage lifetime is increasing it might not be so bad. Or compression/deduplication. As long as they have consistent ad revenue, they’ll be ok.
  - flatiron 4y ago
    I wouldn’t believe dedupe would be a big thing with social media user created content.
    - deleted 4y ago
      [deleted]
    - vineyardmike 4y ago
      I suspect (but have no data for) that a significant chunk of content on social media is memes and still-image-video. Those are likely to be easy to dedupe or compress aggressively
- Spooky23 4y ago
  I disagree, I’m still in the datacenter storage business and our service delivery costs have been going down. You have to look at the service as a whole… nvme let us retire frames, etc. I think hyperscale margins are high, and I think between the many tiers of storage and backend technology they are constantly cooking cost out.
alpark3 4y ago
What happened in 2012 to bump the hard drive prices?
- erichurkman 4y ago
  Floods in Thailand. https://en.wikipedia.org/wiki/2011_Thailand_floods#Damages_to_industrial_estates_and_global_supply_shortages https://en.wikipedia.org/wiki/2011_Thailand_floods#Damages_t...
Rodeoclash 4y ago
Theirs other options available depending on your risk appetite. For example, I built a file sharing tool (https://www.fileyeet.io/ https://www.fileyeet.io/) off the back of Storj (https://www.storj.io/ https://www.storj.io/) which is a distributed file storage backed by a crypto coin (maybe one of the few legitimate uses of crypto, although I'm not convinced yet). Storj was a much cheaper option than S3 although I do have to trust that their systems are as secure as the advertise them to be. Likewise, R2 seems like a good "in-between" option. Both R2 and Storj share the S3 API for integrating with them.
- i5heu 4y ago
  I love Storj, very nice idea. Also they not need cryptocoins to work, they could just replace the tokes with something else.
luhn 4y ago
AWS's pricing strategy changed a few years ago. Previously, they'd drop prices aggressively with much fanfare every time they did. Nowadays, they almost never drop prices, but release new offerings they say will save money [for certain use cases]: - Gravitron 2 is slightly cheaper than Intel instances, with a greater price/performance ratio—at least for some applications. - m6i is same price as m5, but new generation of CPU and increased network performance. - gp3 is 20% cheaper than gp2 and has better baseline performance. It can also scale performance independently of size, so no more overprovisioning storage to hit a certain IOPS. - S3 has IA, Glacier, Glacier Deep Archive, all offering cheaper storage but more expensive retrieval. Not defending AWS here, just noting that they don't seem to be interested in direct price reductions anymore. (EDIT: Previously I stated gp3 and gp2 were the same price. Thanks zerocrates and maxxam for the correction.)
- zerocrates 4y ago
  I seemed to remember gp3 being (incrementally) cheaper than gp2... yeah, gp3 is 8 cents per GB-month vs 10 for gp2 in us-east-1. That's actually a reasonably significant price difference, now that I look at it.
  - pwarner 4y ago
    There were use cases with gp2 where one would over provision capacity to get IOPS. gp3 let you just dial up IOPS which was much cheaper than over provisioning. This may seem a corner case, but it was moderately common. Also AWS should cut their prices, well in this environment maybe holding them is a cut....
- maxxam 4y ago
  gp3 are 20% cheaper than gp2 per GB
- TimTheTinker 4y ago
  This is how every market player is incentivized to behave, apart from regulatory authorities acting to prosecute monopolistic behavior. 1. Price and compete aggressively to grow market share, put competitors out of business, and even transform the market, if possible 2. Once you become the dominant player (i.e. a monopoly), raise prices (or don't lower them) as much as possible without damaging your market position This is why it's so important, especially in the US, for government to begin aggressively enforcing antitrust law again -- something we've failed to do against companies founded since the 1980s. Those laws are there for a reason - not just as a weapon against "evil" (though Standard Oil was admittedly pretty bad) but to keep the market healthy and growing. There are so many positive second- and third-order effects that occur only when competition is healthy and well-regulated, and when prices are transparent.
TedShiller 4y ago
Also need to take into account decreasing cost of bandwidth. This article only mentioned disk storage as an underlying cost.
ocdtrekkie 4y ago
This is eventually why every cloud migration will eventually get undone: As cloud providers dial up the profit margins, going on prem will be a no-brainer.
ksec 4y ago
If you account for IOPS and not just on Cost / GB. The pricing structure of HDD hasn't changed a bit in years. Especially when S3 ( Blue Spot ) used to be way above HDD Cost Per GB price. So while I do agree with AWS pricing structure has changed in recent years. I dont see how the Data shown correlate to that conclusion.
chrislusf 4y ago
Besides storage cost, S3 API access cost can also be high if frequently accessed. And latency is unpredictable. You can use SeaweedFS Remote Object Store Gateway to cache S3 (or any S3 API compatible vendors) to local servers, and access them at local network speed, and asynchronously sync back to S3. https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remote-Object-Storage https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot...
sonnyblarney 4y ago
Egress (primarily) and also S3 costs are the primary reason we look away from AWS for alternative.
skywhopper 4y ago
I’m sure AWS’s margins are growing on S3, but to be fair, S3 is more than just raw storage. The majority of the cost and value is the API and the ever-growing features it provides. Those aren’t getting cheaper nearly as fast as the underlying storage hardware is.
xwdv 4y ago
We currently have plans to move entirely to R2 on our roadmap. Definitely interested in Cloudflare’s offerings.
fnordpiglet 4y ago
Any customer of scale negotiates a deal with aws. Aws loves making pricing deals. Public pricing is advertising.
Helmut10001 4y ago
I cannot speak for the big companies, but for private cloud storage, I suggest to go the route for self-hosting, all the tools are available in 2022: ZFS for reliability, Proxmox for separation of concerns, Opnsense/pfsense for access control, Nextcloud for convenience (if you need such file sync at all). Add a photovoltaic plant and your electricity bill will be _Ok_ (you should do this anyway). I have a 40 TB ZFS Z2 Pool consisting of 6x 8TB drives, and a 16TB offsite pool that is booted for backup snapshots weekly. You'll have to replace the 6 drives running 24/7 approximately every 5 years. If a drive costs $200.00, that will be $1200.00 per 5 years, or $20.00 per month. Add about $400.00 (with PV) to $800.00 (without) for electricity per year ($30.00/$60.00 monthly) and $7.00 monthly for UPS batteries. For these $57.00, you will get a full virtualization feature set under your control, not only a 30TB ingress data sink. With Amazon Glacier, the cheapest "data sink" cloud storage, 30TB would equal $123.00 monthly (or $30.00 with S3 Glacier Deep Archive), with quite a few feature caveats.
- WillPostForFood 4y ago
  I admire that setup, but it could read like an ad for S3. Awesome for a hacker, but too much if you just want a lot of easy reliable storage.
  - RosanaAnaDana 4y ago
    Easy reliable cheap Pick two
- Dunedan 4y ago
  Your setup doesn't factor in labor cost and has a vastly lower durability. That's not to say that S3 is superior compared to your setup, but it's a different solution for different needs.
  - Helmut10001 4y ago
    Yes, of course. Although I weight labor cost and return of technological knowledge gain against each other, i.e. a solution _for different needs_.
aqsheehy 4y ago
I get the feeling the next frontier in storage is freeing up spare capacity. Be that local first or someone cracking efficient p2p storage.
safeerm 4y ago
Like the other comments mentioned your post misses of how transparent AWS has been in price reductions in the past. More importantly, S3 now has several tiers of pricing depending on how frequently you access the data. So maybe lately they haven’t reduced the pricing of the top tier of S3 but they’ve made it significantly cheaper for other use cases of data. That is very contrary to the comments being made of innovators dilemma. (I used to work at AWS but have no knowledge about pricing decisions)

288 points by rckrd 4y ago

all amazon services seem so expensive, hundreds of dollars/ month bills easily for running a few large ec2 servers all month.

snoopy_telex 4y ago
Why haven't there been a upstart taking market share? Why do only the big players get into the game and hold similar costs?
- rckrd 4y ago
  Large fixed costs, economies of scale, whole product (need compute AND storage). It's possible that a company like Cloudflare can disrupt certain verticals in storage that are mispriced by AWS yet have a larger than expected TAM.
- vorpalhex 4y ago
  DigitalOcean, Linode, Hetzner, Vultr...
  - jonatron 4y ago
    DO's spaces didn't even work for the most basic test of putting a few images on a page when I tried it. I couldn't say it's comparable to S3.
    - ipaddr 4y ago
      Something tells me you are doing something wrong.
    - vorpalhex 4y ago
      I use DO spaces heavily - for images no less. It works great. What issues did you have?
      - jonatron 4y ago
        Uploaded about 100 images. Put those images on a page. 50% of them timed out when downloading. Maybe I was just unlucky, but it was a good way to make me instantly lose all confidence DO Spaces. S3 and Backblaze B2 worked fine for the same thing at the same time as when this happened.
        vorpalhex 4y ago
        That's definitely incorrect behavior. Did you reach out to support?
  - upupandup 4y ago
    Which raises my question: Is there some open source github project that lets you create an S3 API compatible with underlying heterogeneous VPS hosts clustered? I'm guessing even with this storage would still be expensive, so how does Backblaze pull this off?
    - vorpalhex 4y ago
      Yes, several. Minio is the big one but I believe Ceph can do this too. Others exist. Disk is cheap. Real cheap.
      - upupandup 4y ago
        ah yes minio! the one issue i have with using VPS for production is, are they hardened? or is it enough just with proper unix user management and UFW? I have this fear that the VPS box has attack surfaces or zero day vulnerabilities but when I am on the cloud I do not have this worry. Perhaps irrational but can't argue with the peace of mind that expensive clouds offer. Although we've seen misconfigured S3 buckets leaking data so.
        vorpalhex 4y ago
        What makes you believe the threat model for AWS is different than DO? They are just VMs at the end of the day.
- waitTho 4y ago
  Human biology is not programmed to abide conflict of interest. On paper things can be written just so, but what’s on paper does not stop feelings, friendship, and connection between two people from forming given biology. Pointing at some philosophy to equivocate away physical laws is the simple con leveraged against the people. The big players leverage their understanding of science and well paid lawyers to play a cognitive game where investment in storage is set aside, as storage is “a solved problem”, they collude to focus government spend on new things they can charge consumers for after charging us via taxes and agency, to build it. Good luck finding a VC willing to compete against Bezos. They’re not going to target the guy managing the infra risk, providing a cheap platform key to their cheap startup gambling. They’re going to target naive college kids to try and build a rocket for them. Because VCs are smarter than Bezos; do none of the work, own the reward.
- 0xbadcafebee 4y ago
  It's easy to enter the market and offer a cheap product. It's hard to enter the market and offer a very solid product. It's nearly impossible to enter the market and provide the hundreds of services the big players can provide, and operate at the scale they can provide, servicing the number of markets and customers they do, with the level of support they do. The big players know that what separates the majors from the minors is trust. If you buy from AWS, you know what you get works, and you will pay a premium for that assurance. And also it is really fricking expensive to be AWS.
  - jamesfinlayson 4y ago
    Very much this - I don't have much experience with providers in this space but Digital Ocean has taken maybe a decade to build and offer a small number of services. That said, there is a lot of AWS that I probably wouldn't know existed - I know in my day job I make big use of maybe five services, plus maybe another 10 glue services between them (CloudWatch, IAM, VPC etc).
- epolanski 4y ago
  Because infrastructure is not that high of cost for most companies and every dev and devops out there knows AWS and has no clue how competitore dashboards even look like.
Sebb767 4y ago
AWS is great if you have lots of money and little time. Which was very true in the last ~decade, where we had lots of venture funded startups that had very much money, but little time. Also, while you definitely pay a markup, the standard EC2 pricing also contains instant availability. If you skip on buying a car and instead pay a taxi to wait 24/7, your costs will also be insanely high. Additionally, AWS provides a great ecosystem where your app can easily be managed in - things like getting a https certificate, setting up a redundant load balancer and even a CDN, database or a Kubernetes cluster can be made simply with a few clicks in an UI. If you don't have someone who knows how to configure those services, it can detract a lot from what you're actually trying to do as a business. Lastly, it has all these enterprise features you suddenly need - solid billing, encryption, certificates etc.. Don't get me wrong, it's expensive, but there's a reason so many businesses use AWS.
chrisan 4y ago
if all you are running are a few ec2 servers then you have quite a lot of competitors to choose from and price out/balance with reliability. we use aws at work, but been running side projects on digital ocean for years for way less than hundreds a month

bicijay 4y ago

Lately im doing Backblaze B2 + Bunny CDN and im very satisfied with the prices

js4ever 4y ago
Blackblaze reliability and performance are below aws, same for bunny cdn. Although I understand it can be interesting for some use cases where perf/reliability is not critical.
- ape4 4y ago
  Counting today's outage? https://news.ycombinator.com/item?id=32267222 https://news.ycombinator.com/item?id=32267222
- upupandup 4y ago
  I guess you get what you paid for but how much of a difference is there? The cost saving it advertises is quite alluring.
  - dilyevsky 4y ago
    Afaik Backblaze has maintenance every Thursday during which they don’t guarantee uptime.
- coder543 4y ago
  Can you cite your sources? AWS "reliability" has been the direct cause of a number of sleepless nights for me over the years. Comparing to a few years ago when I worked on a large-scale product hosted on bare metal servers that worked beautifully, I don't think AWS is all it is hyped up to be. Anecdotal, I know, but even with no experience using Backblaze or Bunny, the bar they would have to meet is a lot lower than you're implying.
  - js4ever 4y ago
    I'm talking about my personal experience, on blackblaze the number of 500 errors was simply not acceptable for my use case, likewise for performance and latency. I was a bit disappointed by bunny cdn rps/latency. But indeed price is not at all comparable. Also I'm not talking about any aws service but more specifically about S3 a d CloudFront. Finally, as I said above those blackblaze and bunny are amazing if you try to optimize the cost as your main goal.

dtx1 4y ago

I suspect it might be an issue of "cheap enough". For smaller Projects, the s3 storage cost doesn't matter too much. Medium sized projects may find the tradeoff between price vs cost of a custom solution to still be more than good enough. Large enough projects simply chose a different, more specialized solution anyway

jenny91 4y ago
Big projects get custom discounts; there's a whole different pricing game for huge customers.
- jopsen 4y ago
  Or they make lots of money and choose not to care.
  - daniel-cussen 4y ago
    There is such a thing. Like if it ain't broke don't fix it, compute costs aren't always that big, sometimes single-thread is enough even on Wall Street exchanges. And like come on originally the alternative was hiring employees and managers for those employees and dealing with human error and deception overlapping with the guilt of exploiting them, the whole management game. Difficult to find true leadership, and a work-ethic shared between manager and employee. And the education I got in the nineties and noughts was made for desk jobs with stationary, not computers. Sometimes automation alone is fine. Even on a computer that hasn't been reset in fifty years and is obsolete according to everyone else, hey if it does the trick. Make sure it stays powered. Cobol. Does the trick. Nuclear plants, they use really old software, one using very new software and connected to the internet was hit by the Morris Worm. I literally picked up a 32 ounce rock on the street that was intended, judging by its shape and way it was cut out of cement and pebble composite, for stoning. Like for Biblical harlots. It was left behind right after a protest was cleared one Friday afternoon on Portugal and Alameda, Santiago Centro, Santiago Chile. Found it Friday like in April, at 22:16. I roam, I checked out the scene--as is my wont--to see what's up, different graffiti on the walls, and then whoops don't see a lot of pint-sized rocks on the sidewalk, somebody might trip, better clear it. Took it home. Realized I should have worn gloves. Next time I went to hang out in front of the police station I told them about it, hey you could do forensics I said, they're like uh no wrong station for forensics, uh...thanks for clearing it away, those are meant for cops. I got stoned with similar rocks after watching cops on motorcycles retreat from a mob, didn't click that mob was throwing rocks, kind of aiming at whatever that moved, and fuck did I then move, I sprinted away to safety before the gates closed on me. Luckily I didn't take a direct hit, none nailed me. But the moral of the story is this: nothing about being tens or hundreds or thousands or in this case millions of years old renders it ineffective as a weapon. Same laws of physics. Same gravity. Muscle equal strength. Rock is just as hard now as it was then. Harder than my skull. Death is bad. Bad then, and bad now.

influx 4y ago

It looks like the most recent data point, S3 was actually cheaper than the raw storage, while providing 11 9's of durability, across multiple AZs. Still looks like a pretty good deal, as the price of storage has mostly flatlined in this graph. Nor does this even account for cold storage, or reduced redundancy.

secabeen 4y ago
The prices are in different units. S3 is $/GB-Mo, raw storage is in $/GB. So, when they are equal on the graph, you are paying each month to AWS what the cost of the raw disk is. Now, yes, you need a lot more than just raw disk to effectively store data, but even if you just assume a 5-year lifespan for that disk, the price difference displayed on the graph as equality is actually a 60x difference in plain $. The price has also appeared to flat-line because they are using a linear graph scale for logarithmic data. It should use a log-scale for the y-axis.
- ceeplusplus 4y ago
  Now factor in redundancy and server costs, and the difference is not so huge. You're still paying a multiple of the raw cost, but unless you're storing on the exabyte scale I think it shouldn't really matter in the grand scheme of things. Cost of a single engineer to manage a Minio cluster probably already outweighs the extra cost you're paying at any reasonable scale (i.e. most companies). And if you're a big player the published costs are not what you're paying.
  - hedora 4y ago
    They use wide erasure code stripes, so redundancy is ~ 1x within a data center. Let's assume 3 data centers. It's well known how to build a storage node whose cost is mostly disks, let's say 50% of the hardware is not disks. So, redundancy and server costs explain up to a 6x mark up. Power and networking really shouldn't account for the other 54x.
    - jenny91 4y ago

Dunedan 4y ago

> Yet, AWS S3 pricing hasn't decreased as fast as the underlying storage costs. > […] > Another blog post analyzes the same theory for compute and finds a similar story using pricing data from AWS EC2. AWS used to do frequent price reductions years ago. At a certain point they seem to have stopped doing that and are now only doing them rarely. That's really a shame as there are still a lot of AWS offerings which are priced way too high (data transfer being the most prominent one). It'll be interesting to see if and up to which point AWS will keep the prices stable with raising inflation.

aabhay 4y ago
One of the benefits of inflation!

epberry 4y ago

The R2 vs S3 story is pretty interesting. For archival use cases, S3 still wins by a mile, but for running apps, R2 often wins (minus lacking features). I wrote up my findings here: https://www.vantage.sh/blog/cloudflare-r2-aws-s3-comparison https://www.vantage.sh/blog/cloudflare-r2-aws-s3-comparison

Dunedan 4y ago

> It's hard to come to a firm conclusion since pricing data is held very secretly (AWS quickly deletes announcements of historical price decreases). That's not correct. Here is the history of every (properly tagged) price reduction AWS ever announced: https://aws.amazon.com/blogs/aws/category/price-reduction/ https://aws.amazon.com/blogs/aws/category/price-reduction/

rckrd 4y ago
Those blog posts are a good start. I was referencing the full pricing page for s3 that breaks down by region and class. AFAIK there are some third parties that track the granular data, but it's not preserved anywhere on the AWS page. A lot of price decreases also might have happened silently (without a blog post) or as de facto decreases (widely but privately negotiated, e.g. "sticker price").
- Dunedan 4y ago
  > I was referencing the full pricing page for s3 that breaks down by region and class. You were referring to the announcements and these announcements are still available. The actual pricing pages of course only show current prices. What would be the motivation for AWS to show outdated prices there as well? No other company I can think of does that and given the complexity of the pricing structure for AWS services that'd only confuse users even more. > A lot of price decreases also might have happened silently (without a blog post) Do you have an example for that or is this just hearsay? > or as de facto decreases (widely but privately negotiated, e.g. "sticker price") Privately negotiated deals aren't regular price reductions as they're only available for customers with a fairly large spend for such AWS services.
  - lstamour 4y ago
    I can say that price _increases_ happened for a few AWS products in specific regions/countries without blog posts. Often AWS simply emails affected customers 1-3 months in advance and after making the change. That said, I imagine prices shifted due to currency, labour costs, energy costs or other regional concerns. And I’ve seen the opposite - I’ve seen AWS freeze prices and eat the cost differences long after they should have raised costs. It varies by product and region and marketing strategy, I expect. The only rule of thumb is what is posted on the pricing page is supposed to be what you get charged. No APIs exist for the most part, and the actual charges can sometimes differ anyway (e.g. grandfathered pricing etc.)

Comevius 4y ago

As a European I take not getting many times as expensive, like our fuel, energy and food prices. I wonder how the European cloud will fare this winter, and how many European customers we will lose.

fxtentacle 4y ago

The main issue we have with S3 is the extortionary egress bandwidth fee. Storage pricing seems OK, but what's the point if I can't send those files to users?

rckrd 4y ago
A bit tangential, but in some cases you can use S3 Gateway endpoints, which can dramatically reduce your egress costs if you're just accessing it within a VPC. [0] https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpo...
- judge2020 4y ago
  This is the entire point of the egress fees. If you can keep traffic inside AWS it's basically free. Getting it out costs money.
tpmx 4y ago
I think this would be an excellent area for regulation since it anti-competitive (portability is dramatically reduced since it has been made artifically expensive to move the data to another cloud service) and the cloud services haven't shown any interest in doing something about it on their own.
- sfe22 4y ago
  Are you seriously suggesting that S3 is a monopoly with means to block competition. I mean even when discussing cloud in general, what you are suggesting is a joke.
  - tpmx 4y ago
    Anti-competitive practices (https://en.wikipedia.org/wiki/Anti-competitive_practices https://en.wikipedia.org/wiki/Anti-competitive_practices) don't require a monopoly.
- ipaddr 4y ago

tick_tock_tick 4y ago

Cloudflare's entry into this market will be interesting no bandwidth costs might put real pressure on AWS to get reasonable with it's pricing. https://blog.cloudflare.com/introducing-r2-object-storage/ https://blog.cloudflare.com/introducing-r2-object-storage/

epolanski 4y ago
That sounds a lot like "let's hope the new AMD GPUs will be priced low so Nvidia will cut their prices"
- tick_tock_tick 4y ago
  I mean they announced pricing already and it's much cheaper.... And are you implying that if AMD prices GPUs cheaply doesn't impact Nvidia?

tnolet 4y ago

For my company, neither egress nor storage cost are the big issue. It’s the API call (PUT) cost. We deal with payloads that are just a little too big for a database (we run Postgres and Clickhouse) but just too frequent (~100 per second) and small (think largish json blobs) to be effective on S3. We are write heavy. Reads are probably 1% but need to be instant for a good UI and API experience.

YawningAngel 4y ago
This is a very naive question so I might be very wrong, but isn't postgres pretty flexible about objects now?
- tnolet 4y ago
  Yes, but it does blow up TOAST and has a lot of impact on the deletion behavior on busy tables. We removed all larger json blobs from PG. typical settings or config stuff in json in PG are fine. We use that all the time. But larger json blobs of several kilobytes are still an issue for semi timeseries data.
  - iknownothow 4y ago
    Could you elaborate on the TOAST issues you're having? We're pretty liberal with our use of large JSONB objects and might hit a billion objects in a year or so.
  - LunaSea 4y ago
    Have you tried using PostgreSQL Large Objects (LOs)?
- michael1999 4y ago
  100/s is 3 billion per year. Postgres has a hard limit of 4 billion blobs in a column. that ignores the reality that RDS storage is comparatively expensive.
  - 4y ago

klodolph 4y ago

It looks like storage costs haven’t changed much since the last S3 price reduction five years ago. My other take on this is that given how slowly HDD costs are going down at this point, tape is going to remain relevant for some consumers for a lot longer than many of us thought.

Sebb767 4y ago

> Even with a slowdown of Moore's Law, it seems like AWS has a healthy margin to continue to offer strategic price cuts only when necessary. Which doesn't surprise me much, really. If your customers are mostly stuck with you, competition is sparse and people pay the price you demand - why would you reduce prices?

advisedwang 4y ago

Hard disks have been getting larger capacity but i/o capability is fairly flat. The result is that price/GB is dropping but price/iops is not. So _cold_ storage is where you would expect to see pricing fall. I don't follows AWS pricing closely but I've seen a lot of news around Glacier over the year so that might reflect this fact.

aledalgrande 4y ago
There's now a new Glacier product, called Glacier Instant Retrieval, which supposedly makes it cheaper and faster to retrieve from Glacier, if you do it a limited number of times.
- pwarner 4y ago
  Yeah this is magic price wise
didgetmaster 4y ago
The capacity and speed curves of drives is quite different. When the first TB drives were released about 15 years ago, the fastest ones were about 100MB/s. Now you can get 20TB drives (20x capacity) and the fastest ones are about 300MB/s (3x speed). It is just far easier and cheaper to make a drive twice as big than it is to make it twice as fast. SSDs are a different animal, but have some similar characteristics. Within the same generation (e.g. m.2 pcie gen 4), you can get drives that have a lot more capacity but have roughly the same access speeds (i.e. the 2TB version is very similar to the 1TB version). The speed increases between generations is much better than with HDD. Pcie gen 3 drives seemed to max out about 3500MB/s while the gen 4 drives are about double that. I have seen reports that gen 5 drives might double it again With HAMR technology we might get HDD drives with capacities in the 50TB-100TB range. You can bet that the speed won't be 5x current technology even if they get dual-actuators in them. There will need to be some kind of breakthrough technology to improve it significantly. This is why we need better data management systems. If the meta-data (e.g. file table) is only 1% of the data that is still a lot of data to read in and store in RAM. We need better systems where the file records are much smaller. https://didgets.substack.com/p/where-did-i-put-that-file https://didgets.substack.com/p/where-did-i-put-that-file

kbumsik 4y ago

Well, at least AWS has not increased the price like GCP did: https://cloud.google.com/storage/pricing-announce https://cloud.google.com/storage/pricing-announce

coding123 4y ago

Because of inflation, the price is going down. However also because of inflation, I have had a 10% "pay cut" since mid last year.

mathgladiator 4y ago

Has anyone had any direct experience between S3 and Backblaze for the basic usecase of uploading and downloading files? https://www.backblaze.com/b2/cloud-storage-pricing.html https://www.backblaze.com/b2/cloud-storage-pricing.html Is just radically different.

dmw_ng 4y ago
B2 isn't great for low latency serving, objects that aren't in hot cache have extremely variable delays on first fetch, and the delay (at least as of a year ago) scales according to object size. For largeish video (over 500mb) I remember seeing >1 second latency, enough to rule out using it for anything public facing

aejnsn 4y ago

> AWS has a healthy margin to continue to offer _strategic price cuts_ only when necessary I’ve never seen a non-strategic price cut. :)

smilekzs 4y ago
I think the contrast is between: A: Cutting prices whenever costs go down B: Cutting prices only when competitors do so
- deleted 4y ago
  [deleted]
fomine3 4y ago
Inflation, virtually!

fleddr 4y ago

Storage price reductions slowing down or even coming to a halt makes me wonder what this means for "infinite storage" companies. I'm talking services like Facebook, Youtube, that are "free" (ad supported) where on a daily basis an absurd amount of new content is added, yet almost nothing ever removed. If storage needs grow endlessly yet storage costs stopped going down, wouldn't that mean that the model in the long term is unsustainable? Sure you can delay the inevitable (compress content, move old stuff to cold storage) but ultimately storage costs per user goes up whilst income likely does not.

prepend 4y ago
If storage lifetime is increasing it might not be so bad. Or compression/deduplication. As long as they have consistent ad revenue, they’ll be ok.
- flatiron 4y ago
  I wouldn’t believe dedupe would be a big thing with social media user created content.
  - deleted 4y ago
    [deleted]
  - vineyardmike 4y ago
    I suspect (but have no data for) that a significant chunk of content on social media is memes and still-image-video. Those are likely to be easy to dedupe or compress aggressively
Spooky23 4y ago
I disagree, I’m still in the datacenter storage business and our service delivery costs have been going down. You have to look at the service as a whole… nvme let us retire frames, etc. I think hyperscale margins are high, and I think between the many tiers of storage and backend technology they are constantly cooking cost out.

alpark3 4y ago

What happened in 2012 to bump the hard drive prices?

erichurkman 4y ago
Floods in Thailand. https://en.wikipedia.org/wiki/2011_Thailand_floods#Damages_to_industrial_estates_and_global_supply_shortages https://en.wikipedia.org/wiki/2011_Thailand_floods#Damages_t...

Rodeoclash 4y ago

Theirs other options available depending on your risk appetite. For example, I built a file sharing tool (https://www.fileyeet.io/ https://www.fileyeet.io/) off the back of Storj (https://www.storj.io/ https://www.storj.io/) which is a distributed file storage backed by a crypto coin (maybe one of the few legitimate uses of crypto, although I'm not convinced yet). Storj was a much cheaper option than S3 although I do have to trust that their systems are as secure as the advertise them to be. Likewise, R2 seems like a good "in-between" option. Both R2 and Storj share the S3 API for integrating with them.

i5heu 4y ago
I love Storj, very nice idea. Also they not need cryptocoins to work, they could just replace the tokes with something else.

luhn 4y ago

AWS's pricing strategy changed a few years ago. Previously, they'd drop prices aggressively with much fanfare every time they did. Nowadays, they almost never drop prices, but release new offerings they say will save money [for certain use cases]: - Gravitron 2 is slightly cheaper than Intel instances, with a greater price/performance ratio—at least for some applications. - m6i is same price as m5, but new generation of CPU and increased network performance. - gp3 is 20% cheaper than gp2 and has better baseline performance. It can also scale performance independently of size, so no more overprovisioning storage to hit a certain IOPS. - S3 has IA, Glacier, Glacier Deep Archive, all offering cheaper storage but more expensive retrieval. Not defending AWS here, just noting that they don't seem to be interested in direct price reductions anymore. (EDIT: Previously I stated gp3 and gp2 were the same price. Thanks zerocrates and maxxam for the correction.)

zerocrates 4y ago
I seemed to remember gp3 being (incrementally) cheaper than gp2... yeah, gp3 is 8 cents per GB-month vs 10 for gp2 in us-east-1. That's actually a reasonably significant price difference, now that I look at it.
- pwarner 4y ago
  There were use cases with gp2 where one would over provision capacity to get IOPS. gp3 let you just dial up IOPS which was much cheaper than over provisioning. This may seem a corner case, but it was moderately common. Also AWS should cut their prices, well in this environment maybe holding them is a cut....
maxxam 4y ago
gp3 are 20% cheaper than gp2 per GB
TimTheTinker 4y ago
This is how every market player is incentivized to behave, apart from regulatory authorities acting to prosecute monopolistic behavior. 1. Price and compete aggressively to grow market share, put competitors out of business, and even transform the market, if possible 2. Once you become the dominant player (i.e. a monopoly), raise prices (or don't lower them) as much as possible without damaging your market position This is why it's so important, especially in the US, for government to begin aggressively enforcing antitrust law again -- something we've failed to do against companies founded since the 1980s. Those laws are there for a reason - not just as a weapon against "evil" (though Standard Oil was admittedly pretty bad) but to keep the market healthy and growing. There are so many positive second- and third-order effects that occur only when competition is healthy and well-regulated, and when prices are transparent.

TedShiller 4y ago

Also need to take into account decreasing cost of bandwidth. This article only mentioned disk storage as an underlying cost.

ocdtrekkie 4y ago

This is eventually why every cloud migration will eventually get undone: As cloud providers dial up the profit margins, going on prem will be a no-brainer.

ksec 4y ago

If you account for IOPS and not just on Cost / GB. The pricing structure of HDD hasn't changed a bit in years. Especially when S3 ( Blue Spot ) used to be way above HDD Cost Per GB price. So while I do agree with AWS pricing structure has changed in recent years. I dont see how the Data shown correlate to that conclusion.

chrislusf 4y ago

Besides storage cost, S3 API access cost can also be high if frequently accessed. And latency is unpredictable. You can use SeaweedFS Remote Object Store Gateway to cache S3 (or any S3 API compatible vendors) to local servers, and access them at local network speed, and asynchronously sync back to S3. https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remote-Object-Storage https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot...

sonnyblarney 4y ago

Egress (primarily) and also S3 costs are the primary reason we look away from AWS for alternative.

skywhopper 4y ago

I’m sure AWS’s margins are growing on S3, but to be fair, S3 is more than just raw storage. The majority of the cost and value is the API and the ever-growing features it provides. Those aren’t getting cheaper nearly as fast as the underlying storage hardware is.

xwdv 4y ago

We currently have plans to move entirely to R2 on our roadmap. Definitely interested in Cloudflare’s offerings.

fnordpiglet 4y ago

Any customer of scale negotiates a deal with aws. Aws loves making pricing deals. Public pricing is advertising.

Helmut10001 4y ago

I cannot speak for the big companies, but for private cloud storage, I suggest to go the route for self-hosting, all the tools are available in 2022: ZFS for reliability, Proxmox for separation of concerns, Opnsense/pfsense for access control, Nextcloud for convenience (if you need such file sync at all). Add a photovoltaic plant and your electricity bill will be _Ok_ (you should do this anyway). I have a 40 TB ZFS Z2 Pool consisting of 6x 8TB drives, and a 16TB offsite pool that is booted for backup snapshots weekly. You'll have to replace the 6 drives running 24/7 approximately every 5 years. If a drive costs $200.00, that will be $1200.00 per 5 years, or $20.00 per month. Add about $400.00 (with PV) to $800.00 (without) for electricity per year ($30.00/$60.00 monthly) and $7.00 monthly for UPS batteries. For these $57.00, you will get a full virtualization feature set under your control, not only a 30TB ingress data sink. With Amazon Glacier, the cheapest "data sink" cloud storage, 30TB would equal $123.00 monthly (or $30.00 with S3 Glacier Deep Archive), with quite a few feature caveats.

WillPostForFood 4y ago
I admire that setup, but it could read like an ad for S3. Awesome for a hacker, but too much if you just want a lot of easy reliable storage.
- RosanaAnaDana 4y ago
  Easy reliable cheap Pick two
Dunedan 4y ago
Your setup doesn't factor in labor cost and has a vastly lower durability. That's not to say that S3 is superior compared to your setup, but it's a different solution for different needs.
- Helmut10001 4y ago
  Yes, of course. Although I weight labor cost and return of technological knowledge gain against each other, i.e. a solution _for different needs_.

aqsheehy 4y ago

I get the feeling the next frontier in storage is freeing up spare capacity. Be that local first or someone cracking efficient p2p storage.

safeerm 4y ago

Like the other comments mentioned your post misses of how transparent AWS has been in price reductions in the past. More importantly, S3 now has several tiers of pricing depending on how frequently you access the data. So maybe lately they haven’t reduced the pricing of the top tier of S3 but they’ve made it significantly cheaper for other use cases of data. That is very contrary to the comments being made of innovators dilemma. (I used to work at AWS but have no knowledge about pricing decisions)