4 ms·
Have you considered putting cloudflare or similar CDN with unlimited egress in front of your bucket? Reading your blogpost I don't fully get how the current si
by flockonus 2y ago
Have you considered putting cloudflare or similar CDN with unlimited egress in front of your bucket?
Reading your blogpost I don't fully get how the current signing implementation can halt massive downloads, or the "attacker"(?) would just adapt their methods to get the signed URLs first and then proceed to download what they are after anyway?
- paxys 2y agoYup. The only mitigation here is that there is a limit to how many different asset URLs they will be able to generate, but if they want to be malicious they can download the same file over and over again and still make you rack up a huge bill.
- dyogenez 2y agoThis is true. I’d still need a CDN in front of the actual files to prevent that. That’s a takeaway for me from this feedback.
- ezekg 2y agoHonestly, I would just move to R2 and save on egress fees even without the CDN. Runaway egress bills are no fun. I saved myself thousands $/mo moving to R2.
- l5870uoo9y 2y ago10k req/s would potentially crash the ruby proxy server halting the image serving. Cloudflare is the way to go. I generally serve heavy files, e.g. videos, from a Cloudflare bucket to avoid expensive bills from primary host.
- frankjr 2y agoYou cannot just put Cloudflare in front of your Google hosted bucket, that's against CF's terms of service. In order to do that you would have to also host the content itself on Cloudflare R2/Images etc. There used to be also html only restriction but that's no longer the case. > Next, we got rid of the antiquated HTML vs. non-HTML construct, which was far too broad. Finally, we made it clear that customers can serve video and other large files using the CDN so long as that content is hosted by a Cloudflare service like Stream, Images, or R2. https://blog.cloudflare.com/updated-tos/ https://blog.cloudflare.com/updated-tos/
- KomoD 2y agoYou totally can, just not a "disproportionate percentage"
- frankjr 2y agoThat's when you're caching a whole page which contains images. The OP is talking about putting the CDN in front of a bucket which doesn't serve anything but images (= 100%).
- kijin 2y agoYou can easily work around that problem by putting the CDN in front of everything, including web pages and API calls, not only your bucket of images. OP might want to do that anyway, since the attacker will now be hammering their Rails app instead of just the bucket.
- JohnMakin 2y agoThis is absolutely nuts to me and would immediately rule out ever hosting anything on google storage for me
- krzys 2y agoIt’s Cloudflare’s, which prohibits usage not directly related to hosting web content.
- dyogenez 2y agoPutting a CDN in front would prevent this at the bucket level, but then someone could still hit the CDN at 10k requests/second. We could rate limit it there though, which would be nice. The downside is that people already have the URLs for existing bucket directly. So we'd need to change those either way. The reason why the attacker couldn't just hit the API to get the signed URLs is due to rate limiting that I go over using the rack-attack ruby gem. Since that's limited to 60/second, that's more like 43k images/day max.
- flockonus 2y ago> someone could still hit the CDN at 10k requests/second CDNs have mechanism to rate limit that you can easily configure, and they will be better at this than a ruby gem (no offence to that). On Ruby you're taking on the rate limiting job down to your CPU and limited visibility per IP... idk man, cloudflare is 20/month.