4 ms·

What stops an attacker from generating millions or billions of requests to your bucket, generating unaffordable bills from the class B charges? When using the

What stops an attacker from generating millions or billions of requests to your bucket, generating unaffordable bills from the class B charges? When using the "free" cache product, solving this is Cloudflare's problem - they have a lot of incentives to stop people from being able to do that as it would otherwise cost them money. However, when using the R2 product, Cloudflare sure would be incentivized to do whatever they can to make all these requests succeed so they can bill you for them. Before someone tries bringing up their rate limiting product, that is more than an order of magnitude more expensive than R2.

vlovich123 4y ago
I think what you’re looking for is Bot fight mode, which is a product we have that lets you block bots and customize how you react to them. There’s a free tier and capabilities get more powerful as you price things out [1]. I would imagine that when we add support for public buckets, we’ll be thinking about how that integrates with Bot fight mode (e.g. enabling the free tier automatically). Putting cache in front of public buckets is also something you’ll be able to do. Rate limiting is very similar to spending limits and, at the scale you’re talking about, they’re just expressing the same concept from different perspectives. R2 is currently naturally rate limited anyway because we can’t go faster but that will obviously evolve as we improve the system. Additionally, you can’t currently have public buckets (at least not without exposing it through a Worker you write). The main use for rate limiting and spending limits is more about being OK choosing to have an outage because of a spike in load (e.g. your project going viral unexpectedly on HN before you’ve cost-optimized it for that kind of scale). From a motivation/incentives perspective, Cloudflare is a SaaS company focusing on security and trust. We’re never incentivized to have unhappy customers, no matter how big of a bill we can deliver for any given month. Doubly so when that comes because you’re under attack vs you’re having success. So TLDR: 1. Bot fight mode is probably useful as a DOS mitigation tool when hosting a public bucket. 2. When we add virtual pathing, buckets are hosted under your account domain. Unlike with other providers, people can’t scrape public records to find what buckets you’ve exposed (intentionally or otherwise) to the public. 3. The product is currently naturally rate limited. 4. Rate limiting isn’t a viable solution here because an outage is not meaningfully better from us charging you insane amounts of month for that month. If you get an unexpectedly large bill, please reach out to Cloudflare support. 5. Public buckets don’t actually exist yet. 6. Cloudflare is in the business of security and trust and we’re incentivized to do our best to engender that trust. Does that speak to your concerns?
- fjfbsufhdvfy 4y ago
  Sounds good! Yeah, my concerns are more related to public buckets, I suppose, and those aren't out. I have some more thoughts on those: - Will adding the cache in front of a public bucket come with the necessary ToS updates to allow serving the media content in the bucket? - It would be nice if there could be some kind of timed authorization for downloads (like a signed token in a header or something?) that does not count towards class B charges if it fails. Currently you can implement this with a worker, but it actually still causes a charge to run the worker, so is mostly ineffective.
  - vlovich123 4y ago
    Just to be clear. Putting cache in front of a public bucket has a lot of technical challenges. I don't know when that will become a thing but presumably when it does we'd figure out what the ToS changes would look like. > that does not count towards class B charges if it fails What does failure mean in this context?
    - fjfbsufhdvfy 4y ago
      Failed to verify the token, as in the user did not have valid authorization to download the file. Something like this would really help cutting down on unwanted bot requests as each valid request could be traced back to a user and abusive users could be banned. But if the "failed" requests still cause a charge on your account, this wouldn't really work as intended. Just trying to think of reliable ways to defend against bot downloading causing massive charges, but also without letting them take down the entire site by triggering something like a billing cap. Hostile world out there...
      - vlovich123 4y ago
        If you gave the user an invalid token (as in R2 can reject it because the token isn't valid), we certainly wouldn't bill (after all, that would be a trivial way someone could launch billing attacks on your account). Now if you gave them a token that validates from our end, then I'm not sure how we'd solve that. I think what you're thinking of is maybe some kind of external IAM integration where you could add an additional authentication layer to whitelabel R2 (but then you're back to the problem where you'd be running a worker to validate the account-level logic and paying for that request). I think the solution is bot fight protection services from Cloudflare. There's a bunch in there also to allow well-behaved bots to have access and lets you control what those look like.

vlovich123 4y ago

I think what you’re looking for is Bot fight mode, which is a product we have that lets you block bots and customize how you react to them. There’s a free tier and capabilities get more powerful as you price things out [1]. I would imagine that when we add support for public buckets, we’ll be thinking about how that integrates with Bot fight mode (e.g. enabling the free tier automatically). Putting cache in front of public buckets is also something you’ll be able to do. Rate limiting is very similar to spending limits and, at the scale you’re talking about, they’re just expressing the same concept from different perspectives. R2 is currently naturally rate limited anyway because we can’t go faster but that will obviously evolve as we improve the system. Additionally, you can’t currently have public buckets (at least not without exposing it through a Worker you write). The main use for rate limiting and spending limits is more about being OK choosing to have an outage because of a spike in load (e.g. your project going viral unexpectedly on HN before you’ve cost-optimized it for that kind of scale). From a motivation/incentives perspective, Cloudflare is a SaaS company focusing on security and trust. We’re never incentivized to have unhappy customers, no matter how big of a bill we can deliver for any given month. Doubly so when that comes because you’re under attack vs you’re having success. So TLDR: 1. Bot fight mode is probably useful as a DOS mitigation tool when hosting a public bucket. 2. When we add virtual pathing, buckets are hosted under your account domain. Unlike with other providers, people can’t scrape public records to find what buckets you’ve exposed (intentionally or otherwise) to the public. 3. The product is currently naturally rate limited. 4. Rate limiting isn’t a viable solution here because an outage is not meaningfully better from us charging you insane amounts of month for that month. If you get an unexpectedly large bill, please reach out to Cloudflare support. 5. Public buckets don’t actually exist yet. 6. Cloudflare is in the business of security and trust and we’re incentivized to do our best to engender that trust. Does that speak to your concerns?

fjfbsufhdvfy 4y ago
Sounds good! Yeah, my concerns are more related to public buckets, I suppose, and those aren't out. I have some more thoughts on those: - Will adding the cache in front of a public bucket come with the necessary ToS updates to allow serving the media content in the bucket? - It would be nice if there could be some kind of timed authorization for downloads (like a signed token in a header or something?) that does not count towards class B charges if it fails. Currently you can implement this with a worker, but it actually still causes a charge to run the worker, so is mostly ineffective.
- vlovich123 4y ago
  Just to be clear. Putting cache in front of a public bucket has a lot of technical challenges. I don't know when that will become a thing but presumably when it does we'd figure out what the ToS changes would look like. > that does not count towards class B charges if it fails What does failure mean in this context?
  - fjfbsufhdvfy 4y ago
    Failed to verify the token, as in the user did not have valid authorization to download the file. Something like this would really help cutting down on unwanted bot requests as each valid request could be traced back to a user and abusive users could be banned. But if the "failed" requests still cause a charge on your account, this wouldn't really work as intended. Just trying to think of reliable ways to defend against bot downloading causing massive charges, but also without letting them take down the entire site by triggering something like a billing cap. Hostile world out there...
    - vlovich123 4y ago
      If you gave the user an invalid token (as in R2 can reject it because the token isn't valid), we certainly wouldn't bill (after all, that would be a trivial way someone could launch billing attacks on your account). Now if you gave them a token that validates from our end, then I'm not sure how we'd solve that. I think what you're thinking of is maybe some kind of external IAM integration where you could add an additional authentication layer to whitelabel R2 (but then you're back to the problem where you'd be running a worker to validate the account-level logic and paying for that request). I think the solution is bot fight protection services from Cloudflare. There's a bunch in there also to allow well-behaved bots to have access and lets you control what those look like.