5 ms·
Just allocate your web crawlers a public IPs and use an internet gateway. Such a weird self-enforced limitation they’ve envisioned, not clear why they think the
by datalopers 4y ago
Just allocate your web crawlers a public IPs and use an internet gateway. Such a weird self-enforced limitation they’ve envisioned, not clear why they think there’s no choice but NAT
- busterarm 4y agoWhy make your web crawlers reachable via the internet? You're one misconfigured security group away from your shit being owned.
- datalopers 4y agoI mean you said it yourself.. that's why you have security groups? You're either paying a ton to use NAT gateway, or setting up a dedicated box to act as your in-house NAT, or configuring security groups. It's pretty simple: no inbound connections except from the VPC. Also even if there's no firewall at all, how does that mean your machines are getting owned? My boxes listen on precisely one port: a heavily locked down sshd (which isn't listening on that interface anyway)
- fuzzybear3965 4y agoI don't understand > listen on one port ... which isn't listening on that interface anyway Would you mind please elaborating, here? And which interface does sshd not listen on?
- datalopers 4y agosshd is only listening on port 22 on the private IP (the VPC) not the public IP of each machine. I then connect into my VPC through a bastion host running wireguard.
- terom 4y agoThat alone will not prevent connections to port 22 on the public IP: the 1:1 non-port-based NAT means that any incoming packets to the public IP will show up at your instance with the private IP as their destination adderess. The TCP/IP stack on your instance knows nothing about the public IPv4 address.
- Bluecobra 4y agoTo elaborate more, in AWS the Internet Gateway modifies the private IP to public IP and vice versa. There’s no public IPs being routed with a VPC, it’s all RFC 1918. When they mention private or public subnets, it just means if the subnet has a route to the IGW or not and if it has a public IP assigned. This was pretty confusing to learn at first.
- Hallucinaut 4y agoNot sure if there is some confusion, or I'm missing the point, but I thought GP's point was clear: the security group would have no inbound rules for 0.0.0.0/0. So the instance would never see the requests unless they originated from GP's internal VPC.
- deleted 4y ago[deleted]
- datalopers 4y agoI don't use NAT gateway. That's why I explained the setup.
- 4khilles 4y agoThe starting point should be the simpler solution. What's the argument for introducing NAT? Why is the firewall (that you need anyways) insufficient?