4 ms·

I immediately groaned when I read "public bucket." On AWS you'd put CloudFront in front of the (now-private) bucket as a CDN, then use WAF for rate limiting, b

I immediately groaned when I read "public bucket." On AWS you'd put CloudFront in front of the (now-private) bucket as a CDN, then use WAF for rate limiting, bot control, etc. In my experience GCP's services work similarly to AWS, so...is this not possible with GCP, or why wasn't this the setup from the get-go? That's the proper way to do things IMO. Signed URLs I only think of when I think of like, paid content or other "semi-public" content.

dyogenez 2y ago
That's a good idea. I probably could've put a CDN in front of this and rate limited there while keeping things public. That might've been faster than using Ruby to be honest. The downside was that our API already shared the non-CDN URLs, so that would leave the problem open for anyone who already had that data.
- arcfour 2y ago
  The bucket is private though, only accessible through the CDN. The old URLs would cease to function. On AWS this is implemented through OAI/OAC, granting the CloudFront distribution access via its own unique principal. AWS has had a baseline security recommendation for years now to disable S3 public access at the account/org level. Maybe this breaks things, maybe you need to expire some caches, but (forgive me for being blunt, I can't think of a better way to say it) that's the cost of not doing things correctly to begin with. My first thought as a security engineer when setting something up to be public has always been "how hard could someone hit this, and how much would it cost/affect availability?"
0xbadcafebee 2y ago
Google Cloud makes it insanely difficult/non-obvious what services you should use to solve these problems (or how to use them, because they're always difficult to use). They have a maze of unintuitive product names and sub-products and sub-sub-products, finding them in a UX is ridiculous, there's no useful tips/links/walkthroughs in the wizards, and their docs are terrible. It's like being trapped in the goddamn catacombs of Paris. On AWS, using buckets with CDN, ALB & WAF are obvious and easy, but on GCP it's a quagmire. The other thing is, AWS WAF was released in 2015, and the Google Cloud Armor WAF feature (the what now?) was released in 2020.
- written-beyond 2y ago
  Honestly this is exactly how I felt about GCP when I was building something that would be used by millions of people. At that scale it's very easy to shoot yourself in the foot and boy does Google make that easy. There were so many things that were outright wrong in their documentation that caused me many sleepless nights. Like not recommending using a pool or closing cloudSQL connections in server less functions because they'll be closed automatically when the instance spins down. Don't get me wrong I had used pools extensively before, and I knew you had to close connections but their docs and examples would explicitly show the connections not being closed, just left for them to close when the instance spins down. Idk why they never thought that an instance might never spin down if it's getting hammered with requests and you end up with hundreds of open connections over multiple instances until GCP starts killing your requests telling you "out of connections" in a server less instance. The vaguest possible error which after a lot of debugging you understand that you can't have more than 100 open connections on a single function instance, but you were technically never supposed to have more than one open at any given time. sigh
antihero 2y ago
That said, if you use CF in front of S3 (which you should), anyone with a gigabit connection can easily cost you hundreds of dollars. I know this because I did this to myself accidentally.
- arcfour 2y ago
  With WAF simple IP-based rate limiting is very simple & cheap. More complex types of limits aren't too difficult either, but even just deploying that is a few clicks.

dyogenez 2y ago

That's a good idea. I probably could've put a CDN in front of this and rate limited there while keeping things public. That might've been faster than using Ruby to be honest. The downside was that our API already shared the non-CDN URLs, so that would leave the problem open for anyone who already had that data.

arcfour 2y ago
The bucket is private though, only accessible through the CDN. The old URLs would cease to function. On AWS this is implemented through OAI/OAC, granting the CloudFront distribution access via its own unique principal. AWS has had a baseline security recommendation for years now to disable S3 public access at the account/org level. Maybe this breaks things, maybe you need to expire some caches, but (forgive me for being blunt, I can't think of a better way to say it) that's the cost of not doing things correctly to begin with. My first thought as a security engineer when setting something up to be public has always been "how hard could someone hit this, and how much would it cost/affect availability?"

0xbadcafebee 2y ago

Google Cloud makes it insanely difficult/non-obvious what services you should use to solve these problems (or how to use them, because they're always difficult to use). They have a maze of unintuitive product names and sub-products and sub-sub-products, finding them in a UX is ridiculous, there's no useful tips/links/walkthroughs in the wizards, and their docs are terrible. It's like being trapped in the goddamn catacombs of Paris. On AWS, using buckets with CDN, ALB & WAF are obvious and easy, but on GCP it's a quagmire. The other thing is, AWS WAF was released in 2015, and the Google Cloud Armor WAF feature (the what now?) was released in 2020.

written-beyond 2y ago
Honestly this is exactly how I felt about GCP when I was building something that would be used by millions of people. At that scale it's very easy to shoot yourself in the foot and boy does Google make that easy. There were so many things that were outright wrong in their documentation that caused me many sleepless nights. Like not recommending using a pool or closing cloudSQL connections in server less functions because they'll be closed automatically when the instance spins down. Don't get me wrong I had used pools extensively before, and I knew you had to close connections but their docs and examples would explicitly show the connections not being closed, just left for them to close when the instance spins down. Idk why they never thought that an instance might never spin down if it's getting hammered with requests and you end up with hundreds of open connections over multiple instances until GCP starts killing your requests telling you "out of connections" in a server less instance. The vaguest possible error which after a lot of debugging you understand that you can't have more than 100 open connections on a single function instance, but you were technically never supposed to have more than one open at any given time. sigh

antihero 2y ago

That said, if you use CF in front of S3 (which you should), anyone with a gigabit connection can easily cost you hundreds of dollars. I know this because I did this to myself accidentally.

arcfour 2y ago
With WAF simple IP-based rate limiting is very simple & cheap. More complex types of limits aren't too difficult either, but even just deploying that is a few clicks.