5 ms·
IMO, the best feature Cloudflare added to R2 was the automatic migration. This makes sense for a CDN to offer, but if R2 works, this makes it pretty simple to m
by marcc 5y ago
IMO, the best feature Cloudflare added to R2 was the automatic migration. This makes sense for a CDN to offer, but if R2 works, this makes it pretty simple to migrate.
"After specifying an existing storage bucket, R2 will serve requests for objects from the existing bucket,"
I've Beene evaluating R2, but this migration path makes it dead simple to use. I just point my code to read/write from/to R2 instead of S3, and I'll get egress fees from AWS during the migration, but then that's it?
- milesward 5y agoinsert doubt
- SteveNuts 5y agoWhat part is OP missing?
- mrweasel 5y agoMaybe the point is that just having the S3 and CDN isn’t enough to move customers. If you’re using AWS, S3 (and Cloudfront) it will just be a tiny part of your infrastructure, why would you move that out of AWS and lose the integration coming from having everything in AWS?
- acdha 5y agoIt might be only part of your infrastructure but on many projects it’s a significant fraction of your total expenditure and storage egress is one of the things limiting use of external services. Having a cheap data access option makes it easy to reconsider those choices when you don’t have to factor egress into the cost of trying an alternative.
- tuananh 5y ago> Double paying for storage is still going to be cheaper than egress fees overall. because of this?
- starfallg 5y agoZero egress costs for assets is a very strong reason for people to move. For public assets, it's a complete no-brainer.
- mrweasel 5y agoI kinda think people will use the S3 proxy feature of R2 for that, which doesn’t really move customers over.
- starfallg 5y agoThen why not store it on Cloudflare itself and save on half the storage, plus less latency? Makes zero sense to keep public assets on S3 in this case. Only when you need ACL integration or glacier would S3 make sense.
- deeblering4 5y agoBecause it's significantly cheaper and diversifies your presence across multiple providers. It's a good idea to split origin and CDN, this way requests can be served from cache while origin is down or overloaded. Plus I'd challenge the suggestions that S3 and CDN are a tiny portion of the infrastructure. For a lot of sites it's a significant chunk, things remaining wouldn't be many, maybe compute, DNS, and CI.
- sokoloff 5y ago> diversifies your presence across multiple providers. In this case, for most AWS/CF customers, that diversification is of the "downtime on either represents downtime for their customer" variety. (I'm still a fan of this offering and the pressure it put on the current high price of egress bandwidth, of course.)
- deeblering4 5y agoIf S3 was down would R2 be down for the customer? If R2 was down could you re-route back to S3? Diversifying infrastructure provides additional options to mitigate outages.
- sokoloff 5y agoIf S3 was down, R2 would be up but the rest of your app on AWS would probably be down in the affected (based on the cascading failures in prior S3 outages) so most AWS users are going to be down. If R2 was down and you didn’t rely on anything else that relied on R2, sure, you could repoint DNS and/or issue S3-direct URLs and then clean it up on the way back out. “Additional options” doesn’t come without “additional complexity” and that complexity has a cost. R2’s a great cost-saving move here; I don’t think it’s a reliability improvement generally.
- outworlder 5y agoThere are many applications that have no dependency on AWS at all, other than S3. This is one of the reasons projects like Minio exist. If you don't care and just need some blob storage, this could be great.
- nawgz 5y agoWhy would you doubt it? Cloudfare is a legit company and a migration workflow like this does not sound like a feat of engineering to accomplish
- gopalv 5y ago> I just point my code to read/write from/to R2 instead of S3 This is probably worse for CloudFront than for S3 itself. Double paying for storage is still going to be cheaper than egress fees overall.
- marcc 5y agoGood point about double paying for storage. There's a long migration problem here to clean up. Most of my buckets have some frequently and some very infrequently accessed objects. To address this, there needs to be some sort of an active migration tool? Does R2 have this built in?
- mcintyre1994 5y agoI’m not sure exactly what you mean by active migration tool, but my understanding is that they copy objects from s3 to r2 on the first request. So if you have infrequently accessed objects they won’t be copied and you won’t be double paying storage until it’s first requested.
- flak48 5y agoJust an idea, maybe the first fetch from S3 should be allowed by R2 do delete the original object from S3 too, so that eventually we're only left with two mutually exclusive sets of files (and no double storage)
- marcc 5y agoI thought about that too. It's could be a good solution because the challenge otherwise is going to be listing all objects in the bucket and comparing to what's in the R2 bucket, right?
- yazaddaruvala 5y agoBased on what I’ve seen in this convo: - Set up the S3 mirror into R2 - Migrate your code to read from R2. - Set up SQS to populate with S3 Create events. SQS listeners just make a GET request to R2 for that file. - Generate S3 events and populate SQS by running List operations or by abusing S3 Lifecycle Management. - Let it process. - Switch writes to R2. This all assumes you can’t delete from S3 till R2 is fully ready. Depending on the application, you could switch over writes to R2 in a different step and also possibly delete the S3 file in the SQS processor.
- 5faulker 5y agoI like Cloudflare for its CDN, but considering its competitors and migration effort I think that only time will tell.
- tommoor 5y agohard same, the automatic migration is a big deal!