16 ms·
Cloudflare R2 storage: Rapid and reliable object storage, minus the egress fees
- rpadovani 5y agoNice, and it seems to be quite cheap as well. It's unfortunate, tho, they don't talk about data residency: where are servers located? Where will my data be copied?
- nanankcornering 5y agoIt's still in development, so no details about that yet. Maybe their top tier DCs? "Our vision for R2 includes multi-region storage that automatically replicates objects to the locations they’re frequently requested from."
- jgrahamc 5y agoWhat would you like the answer to be? Do you need regional control over data storage like we do with this https://blog.cloudflare.com/supporting-jurisdictional-restrictions-for-durable-objects/ https://blog.cloudflare.com/supporting-jurisdictional-restri...?
- rpadovani 5y agoYes, definitely, EU regulated industries need to be 100% sure to keep data in EU datacenters
- eastdakota 5y agoWe plan to add support for data residency requirements on an object-by-object basis. In other words, if you need one object to stay in the EU and another to stay in the US and a third to stay in India you tag each object using the same API and we handle ensuring the residency requirements behind the scenes.
- yawnxyz 5y agowe just got this big grant that works with healthcare and genomic data for a new type of therapy in Australia, so we have jurisdiction requirements (data needs to stay within Australia, some data needs to stay within New South Wales). We're currently talking with your run of the mill providers, but I'd pretty excited to try this out with Cloudflare instead... esp. when previous similar projects have been hit with some nasty egress costs
- Youden 5y agoInteresting pricing considering Backblaze is another Bandwidth Alliance member and they only charge $0.005/GBmonth (vs. $0.015/GBmonth). B2 + CloudFlare gives you a similar deal at a third the cost.
- geostyx 5y agoI'm excited because while B2 + Cloudflare is great, the speed+latency isn't the greatest for some applications. So there's definitely a place for R2 here to compete more with AWS S3 than B2.
- heipei 5y agoI'm a fan of B2 as well, but for some use-cases they seriously need to up their game. They only have three datacenters (CA, AZ, and Amsterdam), they still don't have a public status page, their admin UI is lacking lots of features (like proper invoices and different 2FA options), their permission system is very basic and inflexible, and they are not integrating compute with storage like AWS already does and Cloudflare will eventually be able to. However they are impossible to beat on cost, and for me their latency has recently improved significantly and has become much more stable, so I'm not going to move anytime soon.
- jonplackett 5y agoWhat kind of speed difference are you seeing with B2 vs S3?
- 422long 5y agoLatency (time to first byte) in serving infrequently accessed images was a big problem with me and B2. The cost was low enough that I've stuck with it though and coded on the front end of the site to use placeholder images until the real media can be retrieved.
- gnfargbl 5y agoI recently started keeping about 30TB of ElasticSearch and Postgres backups in Backblaze B2. The price is great, but getting data in is not particularly easy as the Backblaze S3 API seems to fail a high proportion of requests when under load. If R2 can be approximately as reliable on ingest as AWS/GCS/Azure is, but without the egress fees of the other major providers, then $0.015/GB-month seems like a pretty good deal.
- otar 5y agoYes! This is what I was predicting about a week ago on a HN: https://news.ycombinator.com/item?id=28564387 https://news.ycombinator.com/item?id=28564387
- aclelland 5y agoReally interesting, will R2 support lifecycle rules like S3 does? We write around 90 million files per month to S3, if we could replace that with R2 and have the files automatically expire after 30 days that'd be a pretty amazing price reduction for us.
- acdha 5y agoThey appear to be focused on an automatic version: > Behind the scenes, R2 automatically and intelligently manages the tiering of data to drive both performance at peak load and low-cost for infrequently requested objects. We’ve gotten rid of complex, manual tiering policies in favor of what developers have always wanted out of object storage: limitless scale at the lowest possible cost.
- Heliosmaster 5y agoGP isn't mentioning lifecycle to change storage tier, but to delete the files after some time.
- acdha 5y agoYou're quite right — I think I was primed by a different mention of S3's intelligent tiering in this thread.
- greg-m 5y agoHey, I'm the PM for R2. We support per-object TTLs, so this should work!
- ryanworl 5y agoCould you provide some more details on the storage system here? Is it built on Ceph's S3 compatibility? Your durability numbers imply erasure coding. Is that the case?
- dikei 5y ago> Your durability numbers imply erasure coding. Is that the case? Well, you can't really build inexpensive, large scale, reliable storage without erasure coding. So that's probably a given.
- js4ever 5y agoAmazing, this is exactly what I was looking for, an S3 compatible provider with no egress cost ... And reliable. I can't wait to try Cloudflare R2!
- sdfhbdf 5y agoVery exciting. Object storage is getting really competitive and i love the naming scheme alliance - S3, B2 (Backblaze) and now R2, who will do one with a “1”? On a serious note i’m wondering about the signed urls and ACL capabilities of the cloudflare offering cause this is something we use. I’m also interested does R2 replace S3 and CloudFront at the same time? That’d be nice and one headache less.
- sfeng 5y agoThe Cloudflare offering supports their serverless Workers compute which would make signed URLs and ACLs trivial. Cloudflare certainly would also be replacing CloudFront.
- slim 5y agoQ1
- drip-egg-jack 5y ago> Our vision for R2 includes multi-region storage that automatically replicates objects to the locations they’re frequently requested from. Seems like they are going to automatically replicate data to other regions. Something like tebi.io is doing for a long time already, it is a geo-distributed S3-compatible storage that is replicating data across the world to reduce read/write latency and increase speed. If it is done right, this might increase download speeds by a lot especially for big infrequently accessed files.
- StratusBen 5y agoReally curious to see how this goes. If they live up to the following paragraph, that's pretty game-changing: >> This cheaper price doesn’t come with reduced scalability. Behind the scenes, R2 automatically and intelligently manages the tiering of data to drive both performance at peak load and low-cost for infrequently requested objects. We’ve gotten rid of complex, manual tiering policies in favor of what developers have always wanted out of object storage: limitless scale at the lowest possible cost. The amount of effort it takes to understand and account for S3 Intelligent Tiering is somewhat mind-blowing so to get rid of all of that (and the corresponding fees) would be really nice and TheWayThingsShouldBe™ for the customer -- on top of that most users just don't even know S3 Intelligent Tiering exists so it'll be great if Cloudflare just handles that automatically. We at https://vantage.sh/ https://vantage.sh/ (disclosure, I'm the Co-Founder and CEO) recently launched a cross-provider cost recommendation for CloudFront Egress to Cloudflare which was really popular and I can imagine doing something similar for S3 -> R2 once it is live and we are able to vet it.
- blackandsqueaky 5y agoDoes Vantage offer comparisons for Backblaze B2, OVH etc? When looking at object storage, tail latency is probably the single most overlooked metric, and the most material differentiator between providers after bandwidth costs. Don't sweat the cent spent on storing an object, worry about the cost of the 6,000,0000 copies of it you'll ship after it's stored. As for bandwidth, CloudFlare becomes uninteresting the moment your account starts to see any real consumption, even AWS are easier to negotiate with.
- StratusBen 5y agoWe are working a more holistic approach to ingesting and providing recommendations from companies like Backblaze, OVH, etc. in addition to many, many more providers for other AWS services. The goal being that we can give customers the tools they need to get visibility on what options exist and take action themselves from there.
- danielheath 5y agoWhat's "real consumption"? I'm getting 1tb egress / day on a $20/mo plan.
- marcus_cemes 5y agoAs someone who took up making travel videos as a hobby, this is definitely on my radar. Video files are large, although ~20 cents per video streamed for a small website is manageable (S3, Cloud Storage, Azure...), it's the potential for abuse that could drive my bill up that terrifies me, which is why I decided to stick to Hetzner VMs with their 20TB of free egress.
- corobo 5y agoHave you seen their Stream product, any use? Came out of their last batch of features if I recall https://www.cloudflare.com/en-gb/products/cloudflare-stream/ https://www.cloudflare.com/en-gb/products/cloudflare-stream/
- marcus_cemes 5y agoI have, I've also taken a look at Mux. Both would be fantastic options, and I'm still considering them, but I don't have many films and I'm biased towards distributing the original high-quality encodes. Both of these services significantly reduce the file-size with a re-encode, even if they promote an "impercievable quality loss". They seem to be more suited to high traffic on-demand streaming for websites, promotional material, etc.
- thangngoc89 5y agoDo you expect your viewer to download and not view it directly on a website?
- marcus_cemes 5y agoYes, most people I know who have an interest have slow connections, or watch it on a TV with a USB. It's a unique requirement. That doesn't mean they can't stream, there is an option to watch it in the browser, they just need a reliable connection (~30 Mbit/s).
- dkh 5y ago
- dgllghr 5y agoDoes it have read after write consistency? And does it have it across regions?
- greg-m 5y agoYes, we have read after write consistency across regions today. We're considering relaxing that if we need to reduce latency (one of the things to be worked out in beta!).
- zacincceo 5y ago$15 is too expensive, especially if you want to follow proper 3-2-1 it'll cost you $45/TB What happens to your Object Storage buckets when Cloudflare has an outage? - https://filebase.com/blog/what-happens-when-my-cloud-goes-down/ https://filebase.com/blog/what-happens-when-my-cloud-goes-do...
- syshum 5y agoWhat do you mean... The cloud does not need backups /s
- nicoburns 5y agoIf you want to do proper 3-2-1, shouldn't you be storing the other backups on a different service?
- zacincceo 5y agoThat's the reason why most enterprises have 4-7 copies of their data..... No inherent geo-replication by default (and as CF shares, it's a "hotel California" problem.... Too expensive to egress completely from AWS.
- acdha 5y agoThe claim is that the service handles distribution across reliability, so I think the more interesting question is the odds of that _mechanism_ failing when your site would otherwise be up[1]. Similarly, 3-2-1 is a backup strategy and the pricing appears to already include multiple copies using the same mechanism so the correct calculation would be the cost of R2 plus whatever _different_ mechanism you choose for disaster recovery purposes such as on-premise storage or a completely different provider. 1. For example, if you use Cloudflare as your CDN / DNS provider and they have a catastrophic failure, the fact that your storage is inaccessible is just another facet of the same issue.
- jiripospisil 5y ago> Our object storage will be extremely inexpensive for infrequent access and yet capable of and cheaper than major incumbent providers at scale. How frequent is infrequent? In our case it's "never unless other backups fail" and for that S3 Glacier Deep Archive is still cheaper ($0.00099 per GB).
- thangngoc89 5y ago> currently planned to be in the single digit requests per second range This is a little bit above your quoted paragraph
- Wallacy 5y ago"At launch, R2 will cost $0.015 per gigabyte of data stored on the service, about half as much as AWS charges for customers on its frequent access tier. Infrequent access to R2 will be free, the company said. " https://www.protocol.com/enterprise/cloudflare-r2-storage-aws https://www.protocol.com/enterprise/cloudflare-r2-storage-aw... So.... I still need to test, but if that is true can be a game changer. Of course... $0.00099 is almost free, but if everything is done automagically will be awesome.
- lokl 5y agoCan you use R2 to host a static website as easily as you can with S3?
- BlackMiracle 5y agoTo host static websites, you use Cloudflare Pages.
- lokl 5y agoThanks, I am familiar with Pages, but I would like to know whether R2 can also be used for this purpose, in a similar fashion as S3.
- RulerOf 5y agoIndeed, the Browser -> CDN -> Object storage static hosting model all under one roof is extremely appealing.
- segphault 5y agoCloudflare Pages locks you into git-based deployment, which isn't always practical, especially for sites that are heavy with images and other non-text static assets. I don't want to use git to manage the images for my website and I don't want to have to pay Github every month for Git LFS storage and bandwidth costs.
- DenseComet 5y agoYou can take a look at Workers Sites instead. Similar to their Pages solution, but with a lot more flexibility.
- greg-m 5y agoSure, you could just setup a Worker in front of a bucket. This is very similar to how the original Workers Sites worked.
- jl6 5y ago
- BlackMiracle 5y agoCan anyone confirm if R2 must be used with Workers and other Cloudflare services only? Is it possible to use as file storage instead of S3?
- sudhirj 5y agoThey've said it supports the S3 API, so yes, you could use it wherever you'd use S3 instead.
- hartator 5y agoThey say fully compatible with S3 API. I expect workers access to be just a sweet addition.
- greg-m 5y agoYes. We have an in-Worker API to start, but will also make R2 available outside of Workers as an API endpoint.
- dhess 5y agoWe have a fairly large Nix binary cache in S3, with not insignificant egress fees, and I'd love to try this out as a replacement.
- jlelse 5y agoCurious to see how this compares to Bunny.net Edge Storage (they are working on S3 support as well): https://bunny.net/edge-storage/ https://bunny.net/edge-storage/
- sudhirj 5y agoI know the Cloudflare team hangs out here, so thanks, and great job! This was absolutely necessary for my line of work. Couple of quick questions/confirmations: * R2 will support the same object sizes as S3? We have 500GB+ objects and could go to a 1TB per object. * R2 will support HTTP Range GETs, right? Egress bandwidth for objects on S3 is the biggest line item on the AWS bill for a company I work for, by an order of magnitude, and this will just wipe it off for the most part.
- greg-m 5y agoHey, I'm the PM for R2. Yes to range requests. Current object limit is smaller than that, but we don't have a fundamental restriction there. Shoot me an email to gmckeon [at] cloudflare.
- mnw21cam 5y agoI'd say definitely increase that limit. I have a 4TB file hanging around over here, I'm sure others have bigger ones.
- greg-m 5y agowe will :) we're working to reach 5 GB per upload, with multi-part support.
- danielrhodes 5y agoI personally think uploading massive files is not usually desirable. Better would be an easy way to chunk it and upload and have the server put the file back together, which would increase reliability.
- zackbloom 5y agoNative tus support would be pretty amazing.
- alberth 5y agoThis is fantastic, I only need now a Cloudflare RDMS to run my entire business on Cloudflare. (Workers KV is great but there's a ton of times that you just need an actual relational database)
- xrendan 5y agoDepending on your use-case (if it's read heavy), you could use SQLite hosted on R2 being read by workers
- greg-m 5y agoDurable Objects is as close as we come today, but we still have two innovation weeks left this year.
- alberth 5y agoReally hope you’re a Cloudflare employee, and that’s a teaser :)
- aenis 5y agoSounds great (nearly too-good-to-be-true great). Wonder how the SLA will look like. I have been using gcs, s3 and firestore - and their actual reliability varies significantly, while advertised slas are similar. For instance, with firestore one has to implement a pretty lenient expotential backoff in case of a timeout, and if the backoff results in the object being retrieved in, say, 2 minutes -- thats still ok as per gcs sla. It obviously makes it hard to use it for user-facing stuff, such as chatbots, where you can't afford to wait that long. In my anecdotal experience of using firestore for about 10 million operations per day, we will usually have a problem like that every few days, and that means user-noticeable failure. It would be great to read more on cloudflare's approach to reliability defined as "99%" percentile max latencuy. Can't wait to give it a try with our workloads.
- the8472 5y ago> R2 is designed with redundancy across a large number of regions for reliability. We plan on starting from automatic global distribution and adding back region-specific controls for when data has to be stored locally, as described above. Does that mean automatic caching across regions? Low-latency read access everywhere without an extra CDN in front of it?
- greg-m 5y agoHey, PM for R2 here. We're still deciding on how we want to handle caching. We integrate with Workers, so manually caching is always possible. The catch is we're currently building for strong consistency - if we added a cache in front, we'd weaken that - so it will likely be a configuration option we add later.
- js4ever 5y agoI'm more interested in eventually consistent with low latency. Please add this setting ASAP :)
- drip-egg-jack 5y agoCheck tebi.io - it is a geo-distributed S3-compatible storage that does exactly that. You can configure global consistency level for each bucket individually.
- wiradikusuma 5y agoFor those who don't want to wait, there's DigitalOcean Spaces (https://www.digitalocean.com/products/spaces/ https://www.digitalocean.com/products/spaces/). Disclaimer: I haven't used it, but planning to, since I already use their VPS.
- slig 5y agoScaleway also provides a similar service. Had an issue recently (deleted a bucket, and couldn't create another bucket with the same name) and their support replied back in minutes.
- frakkingcylons 5y agoNot that Spaces isn’t a good product, but there are countless S3 compatible object storage services. This announcement is noteworthy because there’s no egress fees. Spaces still charges 1c per gigabyte for egress. Not comparable in that aspect.
- numair 5y agoLove the team at DigitalOcean but Spaces was NOT reliable the last time I played with it. It’s also lacking key features in lifecycle and notification areas. Maybe they’ve gotten things together in recent months/years and I haven’t gotten the update, but it was very “you get what you pay for” when I tried to stage a project using it in 2018/19.
- zeitg3ist 5y agoSpaces is not good, in my experience. Their Frankfurt storage just stopped working for like a week. And last time I checked they still didn't support critical features like per-bucket access keys or index.html support.
- pier25 5y agoThis is really a game changer for storage. Well done Cloudflare. How will other providers respond to this now? AWS, GC, and others do not really pay for egress charges themselves. Those super high egress charges are pretty ridiculous.
- js4ever 5y agoYes and I hope this will force AWS & co to end this egress madness
- Shadonototra 5y ago> $0.015 per GB of data stored per month we are still living in 2007?
- frakkingcylons 5y agoYou’re misreading that as $0.15 per GB.
- numair 5y agoThere is one major reason S3 remains the king of storage for mobile media uploads: bucket notifications. Does R2 implement this feature? If so, I’m going to have to run some experiments with this...
- sudhirj 5y agoTo be honest, I think the biggest draw will be for companies (like where I work) that put large objects on S3 and distribute it to hundreds / thousands / millions of customers. The egress direct from S3 is on the order of 8 cents a GB, and with Cloudfront in front of it it’s a few cents lower, and you can negotiate pricing a little lower if you’re big enough. But not an order of magnitude. We’d stick R2 in front of an S3 bucket and wipe off a the biggest portion of our bill. S3 does absolutely have a ton of other stuff like events, lifecycle, Glacier, Lambda etc and is plugged into the AWS ecosystem, so I doubt we’re exiting it completely. But this is a solid option as a front for egress from S3.
- greg-m 5y agoHey, PM for R2 here! We're fully integrated with Workers, so you can write a Worker that calls additional logic when a request is made to the bucket's url. We have notifications, where a change to the bucket invokes a worker with a specific event, on our roadmap, but have a bunch of other work prioritized ahead of them.
- numair 5y agoHi Greg, thanks for the feedback. It’d be great if you could get around to putting up an example of how to do bucket notifications + lifecycle using Workers as a temporary workaround until it’s part of the “core.” I don’t think I’m the only person with this use case, but maybe I’m more of an edge-case minority than I imagine... In any case, a code library / “recipe collection” (do we still call them cookbooks?) would be great when this launches.
- throwaway984393 5y agoI know this is irrational and not very helpful, but it actually makes me angry when I see a response like "but you can cobble together some janky custom code with Workers to do what you want". It's like going to a restaurant, asking if they have shrimp scampi, and getting a reply that you can go to a supermarket and buy the ingredients and make the dish and bring it back to the restaurant to have with your meal. Just tell me you don't have shrimp scampi.
- sneak 5y agoWhat's the censorship policy? Is this going to be content-neutral, like Cloudflare was when fronting ISIS websites? Or is this going to be fine-until-bad-PR, like when Cloudflare decided to stop hosting The Daily Stormer? There is a special kind of lock-in when it comes to object storage, as generally you use something like this when the data is too big to store another copy of locally or at another provider. It's not like you can easily maintain provider independence, and if Cloudflare decides one day that some of your UGC in a bucket isn't something they want to host, what happens then? Is the data lost forever because your account is nuked? Is there a warning or grace period? I am hesitant to put any large amount of data into a service without a crystal clear statement on this, so that I can know up front whether or not a business needs to maintain a second, duplicate object store somewhere else for business continuity. If Cloudflare in practice is going to nuke the account the moment your site ends up hosting something objectionable, this DR requirement (a second provider that also stores all objects) needs to be factored into a customer's costs. (It may be that the bandwidth savings still make it worth it to use Cloudflare even with double storage.)
- chromatin 5y agoWell, it is in feature preview so this is probably not defined. You raise an interesting point regarding object storage being a relatively unique type of lock-in.
- celsoazevedo 5y ago> I am hesitant to put any large amount of data into a service without a crystal clear statement on this, so that I can know up front whether or not a business needs to maintain a second, duplicate object store somewhere else for business continuity. It's a mistake to rely on a clear statement when you can't afford to lose your data. Stuff happens all the time... mistakes, malware, an expired credit card, etc. Independently of the provider you decide to use, I'm not sure if a backup is optional in your case.
- sneak 5y agoFeature request: automatic ipfs integration.
- deleted 5y ago[deleted]
- xfalcox 5y agoI have been doing work on ensuring my app is compatible with the many Object Storage services. Would be great to get access to this early and make it compatible too.
- bentaber 5y agoSo the naming strategy is to call it R2, because it comes before S3? I like it
- vira28 5y agoMatthew tweeted "Why R2? Because it’s S3 minus the one most annoying thing: egregious egress." https://twitter.com/eastdakota/status/1442879872154566658 https://twitter.com/eastdakota/status/1442879872154566658
- zoso 5y agoSeems like a cheap option to backup my photos (just need to think about the reliability/upload multiple times to different regions).
- cloudfalcon 5y agoHaving done hundreds of TCO analyses for customers moving object storage providers, this seems like it carves out a very interesting niche for Cloudflare. R2's higher storage costs (roughly triple) also make it a more manageable threat to specialized solutions like Storj DCS, Wasabi and Backblaze B2. At Taloflow (https://www.taloflow.ai https://www.taloflow.ai), (disclosure: I'm the CEO/Cofounder) we provide buying insights for cloud object storage (and soon other IaaS/PaaS). We will definitely be adding Cloudflare R2 to the mix.
- ignoramous 5y ago> As transformative as cloud storage has been, a downside emerged: actually getting your data back... When they go to retrieve that data, they're hit with massive egress fees that don't correspond to any customer value — just a tax developers have grown accustomed to paying. Strategy Letter V, commoditize your competitor's advantages! > We’ve gotten rid of complex, manual tiering policies in favor of what developers have always wanted out of object storage: limitless scale at the lowest possible cost. Cloudflare has a clear strategy: Be the simplest cloud platform to deploy to. It has been a breeze as a small dev shop adopting their tech. AWS started with the startups, but have since long struggled to keep up that simplicity in face of supporting what must be a dizzying array of customer requirements. Remains to be seen how Cloudflare fares in that regard. I like my Golang better than Rust. > Cloudflare R2 will include automatic migration from other S3-compatible cloud storage services. Migrations are designed to be dead simple. Taking a leaf out of Amazon Data Migration Service and its free transfers from elsewhere into RedShift/RDS/Aurora/OpenSearch. Niice. > ...we designed R2 for data durability and resilience at its core. R2 will provide 99.999999999% (eleven 9’s) of annual durability, which describes the likelihood of data loss... R2 is designed with redundancy across a large number of regions for reliability. S3 goes upto 16 9s with cross-region replication... and so wondering why R2's still at 11 9s? May be the mutli-region tiering is just acceleration (ala S3 Accelerated Buckets) and not replication? > ...bind a Worker to a specific bucket, dynamically transforming objects as they are written to or read from storage buckets. This is huge, if we could open objects in append-mode. Something that's expensive to do in S3 (download -> append -> upload) even after all these years. > For example, streaming data from a large number of IoT devices becomes a breeze with R2. Starting with a Worker to transform and manipulate the data, R2 can ingest large volumes of sensor data and store it at low cost. Okay, where do I sign-up? > R2 is currently under development... Oh.
- fideloper 5y agoBeing able to append to an object would be amazing!
- whalesalad 5y agoI'm trying to figure out the long-con of Cloudflare. Too many free/fantastic products.
- elnygren 5y agoIt certainly ain't free or cheap if you are running a real business with real traffic and real customers. However, still totally worth it, easily. Just saying they have a healthy B2B business model.
- corobo 5y agoGet nerds in now with cool cheaper stuff, later when they're enterprise level decision makers Cloudflare is burned into their brain as the go-to Of course the other angle is that Cloudflare one day ends up like Google, who ended up like Microsoft, who ended up like IBM As for right now though, cooooool
- 2Gkashmiri 5y agouh,, i have to ask. how about using this R2 to host peertube content? it would be somewhat expensive but you are not concerned with egress?
- ttobbaybbob 5y agoNice to see someone flipping the script and encroaching on AWS' territory rather than vice a versa Taking the have-a-much-better-product route to siphoning use from AWS is particularly ambitious. I hope it works out. AWS has had it a little too easy for too long
- drip-egg-jack 5y agoActually many companies are trying/tried to compete with AWS especially in an egress price territory but because of a lack of exposure it is very hard to get a traction and get big enough. And there is a lot of things that can be improved - like egress price, better support, get rid of regions completely, etc...
- rsync 5y agoThis is interesting ... For the longest time we tried to convince people that they should have an off-amazon archive of their S3 data ... we even ran an ads to that effect in 2012[1]. The (obvious) reason this isn't compelling is the cost of egress. It's just (relatively) too expensive to offload your S3 assets to some third party on a regular basis. So if R2 is S3 with no egress, suddenly there is a value proposition again. Further, unlike in 2012, in 2021 we have really great tooling in the form of 'rclone'[2][3] which allows you to move data from cloud to cloud without involving your own bandwidth. [1] The tagline was "Your infrastructure is on AWS and your backups are on AWS. You're doing it wrong." [2] https://rclone.org/ https://rclone.org/ [3] https://www.rsync.net/resources/howto/rclone.html https://www.rsync.net/resources/howto/rclone.html
- slig 5y agoThe [3] resource is fantastic! Have you tried sponsoring rclone? I was studying their docs last week, and I'm sure people reading the docs are interested in this use case of moving between clouds without using their own bandwidth.
- rsync 5y agoThanks - I made a point to have that document created not specifically with rsync.net use-case in mind. You can follow that howto with any two cloud endpoints if you wanted to.
- blackandsqueaky 5y ago> So if R2 is S3 with no egress, suddenly there is a value proposition again. That doesn't appear to be what they're doing, they don't seem to have changed their existing operating model at all: > R2 will zero-rate infrequent storage operations under a threshold — currently planned to be in the single digit requests per second range. Above this range, R2 will charge significantly less per-operation than the major providers. Our object storage will be extremely inexpensive for infrequent access and yet capable of and cheaper than major incumbent providers at scale. What I read this as is "we won't bill you until your traffic spikes, then you'll pay us, oh how you'll pay us" Transparent bandwidth pricing would be a far more interesting announcement. This is the second post I've seen from CloudFlare in recent months throwing bricks at AWS over bandwidth pricing, while failing to mention CloudFlare bandwidth is some of the most expensive available.
- thecodrr 5y agoNot sure why this is more appealing than Wasabi? As far as I can see, Wasabi is cheaper, has great speeds, fantastic S3 compatibility, their dashboard is a joy to use so what is the actual "special" thing here? I mean sure, it's a good thing to have more competition but the way everyone here is describing the situation makes it seem as if Cloudflare is going to be the cheapest & the best.
- NicoJuicy 5y ago1 - Their portfolio is bigger. Making it easier for existing clients to adopt. Eg. Why a lot of companies use AWS. This product seems a good fit for their cloudflare workers. 2 - Cloudflare is going heavy on egress costs. Which they have a lot of extra opportunity costs, comparible to the big 3 cloud providers and they are putting their "weight" in good use: https://blog.cloudflare.com/bandwidth-alliance-charges-forward/ https://blog.cloudflare.com/bandwidth-alliance-charges-forwa... Egress/ingress costs is also important for pricing fyi. 3 - Cloudflare has a known good reputation for their services ( eg. Uptime and costs) + an active community. ( Eg. Check their blog ) Tbh. I didn't knew Wasabi, will check them out though. 4 - Their datacenters are at the edge. Hard to compete on that physical advantage in terms of speed. Ps. Yes, I could be considered a fan, similar to AMD in the past ( still do fyi).
- ukd1 5y agoWasabi is actually pretty great. +1 on this.
- jjeaff 5y agoMy understanding with Wasabi is that it is only intended for up to 100% egress. It doesn't look like r2 has this limitation. Though I can't imagine egress is actually completely free beyond a certain point.
- macromaniac 5y agohttps://wasabi.com/rcs-pricing-faq/ https://wasabi.com/rcs-pricing-faq/ >If your monthly egress data transfer is greater than your active storage volume, then your storage use case is not a good fit for Wasabi’s free egress policy >If your use case exceeds the guidelines of our free egress policy on a regular basis, we reserve the right to limit or suspend your service Unless I am misreading, wasabi can shut you down if your egress is high.
- RobLach 5y agoImpressive carve-out in the object storage market.
- rad_gruchalski 5y agoThis is fantastic. This + workers has so much potential to shake up the market of serverless compute!
- abhv 5y agoVery generous offer from Cloudflare. I signed up. The main question: how can Cloudflare make this into a sustainable business? * cost/gb is cheaper or same as s3, gcp, azure * no egress charges to customers, but they still have to pay for transit when they cross an AS! what is the hidden angle Cloudfare is using here?
- NicoJuicy 5y agoI don't know/think there is a hidden angle. Eg. What could be the hidden angle of providing domains at cost ( which they do). I think it's a matter of having a more complete cloud suite and expanding their portfolio while gaining market share. Clarification: I don't think they do this at cost. Just cheaper than the big 3.
- abhv 5y agoI suspect the story is a bit more complicated. They seem to learn a lot by centralizing a lot of the internet's traffic through their core. Perhaps it makes their ddos-protection more robust when they can train on "what normal usage" looks like "for the longer tail". Perhaps they are building moats around that business and expecting a future when Fortune1000 requires their cdn.
- NicoJuicy 5y agoI don't think that's correct. Their CDN is based on their SDN. Which is also at the core of all their products, including DDOS protection. Their primary product is the SDN and a CDN was a logical pick as first product "release". Eg. You shouldn't consider their products in "regions" as traditional cloud providers. In cloudflare it's programmable, not configurable.
- wongarsu 5y agoCost/GB is three times that of Backblaze, bandwith is generally too cheap to meter (outside aws, gcp, azure who use egress costs for lock-in). The offer is no doubt competitively prized, but it's no doubt much more lucrative than people using Cloudflare as a CDN for their s3/b2/azure/whatever.
- fabian2k 5y agoAm I reading this right that this is pretty much aiming to be a full competitor to S3? Or is this a more limited special purpose tool? I didn't really follow what Cloudflare is doing, so I only know them as a CDN. Are private buckets possible with this? We all knew that the big players are really maximising their profits on the egress charges, so I can see that this left some potential for someone to step in. No egress charges at all still sound a bit too good to be true, but it would be nice as that's just one parameter less to think about. Another interesting aspect are CloudFlare Workers. As far as I can tell they're not a full replacement for something like AWS Lambda if e.g. I need to do a bit heavier stuff on the data in R2. Being able to do heavier processing close to the actual data would be really interesting as well.
- barefeg 5y agoWhat about the AWS bandwidth costs of accessing cloud fare’s data over the internet?
- irq-1 5y agoRequest: Use workers KV to coordinate object locks, like a file system. (Well, something better than a file system.)
- yencabulator 5y agoYou can do that right now with Durable Objects.
- fjfbsufhdvfy 5y agoWill we be able to serve files from a R2 bucket through a subdomain like static.project.com/filename WITHOUT using a worker that wastes money on every request for no reason?
- kondro 5y agoI’m curious what read and write request prices will be. Like egress pricing, S3 starting at $5/million writes and $0.40 per million reads feels excessive.
- orf 5y agoI see the PM for this product is here, a few things we find useful with S3: - Daily inventory report of the contents of the bucket in parquet format - Signed URLs - Lifecycle policies based on tags (but to be honest just a policy that isn't restricted to suffix/prefix would be amazing) - Bucket policies (restrict object key structure) Lots of these work well enough in AWS but are often lacking in some regards with annoying restrictions. Looks like an amazing product, good luck!
- truetraveller 5y ago>providing zero-cost egress for stored objects — no matter your request rate What's the catch? Imagine a few cases. Let's assume s3 volume rate of $50/TB. -I post 1gb video file on Reddit. 100k downloads / month: $5k -I make a 1gb desktop app. I have 100k downloads / month: $5k -I post 100gb data file on Github. 10K downloads / month: $50k. Would I pay $0 on R2? And would there be throttling / rate-limiting? [Edit: Added more realistic examples]
- Strom 5y agoGetting 100k people to download 100GB seems like a rather lofty goal. Has this ever been done via social media? The only case I can think of is a game like Call of Duty, which has a ginormous marketing campaign behind it. Even so the download wasn't linked via social media. Most people don't even have 100k followers. The follower-to-click ratio isn't going to be 100% in any situation. Click-to-full-download ratio is going to be low too, especially so at 100GB. A lot of people don't have that much free space on their devices even! I think this scenario is firmly in outlier land, and thus not really relevant to cloudflare's calculations.
- hakube 5y agoIs Cloudflare working it's way to be a cloud service competitor to AWS or other services out there?
- lewisl9029 5y agoWow, with global replication by default, this looks absolutely perfect for what I'm currently building, even before taking costs into account. I'm hoping this means what I think it means, that write latencies will be minimal across the globe, since writes will be persisted and ack'd at the closest region and then eventually consistently propagated to other regions? If so, curious what would happen in a scenario where a region requests an object that has been persisted at another region but not yet propagated? Will it result in a 404 or is the system smart enough to route the request to the region that has the file at the cost higher latency? From my research so far into S3's cross region replication, the latter behavior doesn't seem possible out of the box since requests have to specify a single region (S3 experts, please do correct me if I'm wrong), so I'm hoping CloudFlare with its deep expertise in managing a global network can differentiate here. Even if it's not offered out of the box, due to the lack of egress costs, it's a lot more feasible to build in this behavior in the application layer with R2 by just racing requests across several regions and taking the one that resolves first (or at all), so very promising regardless. Also, would love to hear some numbers on what kinds of write latency to expect. From my experience so far, S3 writes for tiny files in a single region take on the order of 50ms ish even for clients in close physical proximity, which is serviceable for my use case, but seems higher than it needs to be (and every little bit I can shave off on latency helps tremendously for what I'm building). Really looking forward to seeing what the CloudFlare solution is capable of here. Lastly, S3 didn't advertise and guarantee strong read-after-write consistency for same region read/write until late last year. Will R2 offer this out of the gate?
- ctvo 5y ago> If so, curious what would happen in a scenario where a region requests an object that has been persisted at another region but not yet propagated? Will it result in a 404 or is the system smart enough to route the request to the region that has the file at the cost higher latency? It's eventually consistent for global replication, but additionally there should be a consistent index of things that are available in other regions? I suppose that's plausible. Seems to defeat a lot of what they're avoiding. Nothing stopping you from doing a scatter gather on a 404 base on some heuristic in your application code though.
- 5y ago
- chrislusf 5y agoMost cloud object storage can be a good off-site backup. But accessing your data fast and cheap is not easy. That is why SeaweedFS added a gateway to remote object store. https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remote-Object-Storage https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot... , which asynchronously write local changes to the cloud. If there are enough local capacity, there should be no egress cost. Hopefully, this can change the common pattern, to really treat the cloud object store as a backup.
- neom 5y agoI'm not a software developer so please pardon my ignorance, doesn't this put them at basically a "core complete modern cloud provider" - if you fully bought into architecting against them, with their workers product and all, could you fully build and run for CF?
- maxgashkov 5y agoI think they're still PaaS and not IaaS. So yes, you can build fully-fledged app with them if you're willing to change your engineering paradigm and not use any of the usual arbitrary computing workloads.
- itake 5y agoYou could probably build a lot on their infra and maybe small b2b startups could get by on their offering, but you can easily outgrow their current offering. Azure/GCP/AWS have significantly more tools that solve expensive problems (like ML hosting, events, SQL database, search, etc.).
- alanlbird 5y agoI recently did a pricing comparison of cloud object storage services for my article "How to Create a Very Inexpensive Serverless Database" (https://aws.plainenglish.io/very-inexpensive-serverless-database-6ed6df489ab6 https://aws.plainenglish.io/very-inexpensive-serverless-data...). It describes using object storage as an inexpensive serverless key-value database. Although egress (outbound network) can be a significant part of object storage expenses, if you are reading and writing small objects, per-request expenses can be much bigger. Cloudflare indicates that for low request rates there won't be any request fees, but doesn't state what they will charge for high request rates. My article points out that the best deal when working with high request rates is to use services that don't charge per request such as DigitalOcean, Linode, and Vultr. If it's S3 that you want, even Amazon has recently joined the budget club with Lightsail Object Storage which has monthly plans of $1, $3, and $5 (250 GB storage and 500 GB egress) with no per-request fees.