3 ms·

Personally I blame the irritation and cost of setting up private subnets that can access the Internet via NAT in AWS VPCs in addition to Docker's interactions w

by ary 4y ago

Personally I blame the irritation and cost of setting up private subnets that can access the Internet via NAT in AWS VPCs in addition to Docker's interactions with UFW. Private VPC subnets with NAT Gateways are both unacceptably costly and complicated to setup (source: recently dealt with the combination of NAT Gateways + the Network Firewall service in AWS). Throw in the need for VPC endpoints for many workloads and AWS networking starts to look like an extortion racket for those who want the majority of their infrastructure to be sequestered from the Internet.

lambdasquirrel 4y ago
I’m not sure I’m understanding this entirely. I thought the way you should set it up is have e.g. your nginx / istio gateways exposed to the public internet on EIPs. You shouldn’t have your databases exposed to the world, or the hardware running your backend services directly exposed either. Also, security groups. Yes, if you have to use NAT gateways for serving traffic, those are very expensive.
- ary 4y ago
  What I'm noting is in line with your thinking. Whatever ways AWS marketing might encourage people to do things via their "Well Architected" series of publications, the reality is that the path of least resistance with VPC networking is what the linked article describes: everything on public subnets (with maybe security groups or network ACLs to restrict it somewhat). Splitting infrastructure into public/private subnets is very tedious, and can at times be difficult to debug. In some of my most recent work it took some time to understand why and how the components needed to be linked to make a combination of an Internet Gateway, Network Firewall, and NAT Gateway work together. Combined with the use of multiple private subnets residing in different AZs meant testing permutations and fussing with the "reachability" analyzer. It is this kind of thing that makes people go with the first thing that works so that they can move on to getting productive work done. Typically one wouldn't serve traffic via a NAT Gateway, and that's true for our case as well. The problem we ran into was not properly configuring VPC endpoints for a workload that heavily interacts with S3 and running up a tiny, but still not ideal, bill for network utilization over the NAT Gateway instead. It's not obvious to may people that interacting with AWS APIs happens over the public Internet even from within AWS infrastructure. The cost of ongoing cost NAT Gateways versus the free per-VPC Internet Gateway feels like it disincentivizes starting with a more secure setup. Ultimately what I'm trying to say is that it's somewhat negligent of AWS to not make an almost universal requirement of modern web and SaaS cloud networking be simple and cheap to implement.

by ary 4y ago

lambdasquirrel 4y ago

I’m not sure I’m understanding this entirely. I thought the way you should set it up is have e.g. your nginx / istio gateways exposed to the public internet on EIPs. You shouldn’t have your databases exposed to the world, or the hardware running your backend services directly exposed either. Also, security groups. Yes, if you have to use NAT gateways for serving traffic, those are very expensive.

ary 4y ago
What I'm noting is in line with your thinking. Whatever ways AWS marketing might encourage people to do things via their "Well Architected" series of publications, the reality is that the path of least resistance with VPC networking is what the linked article describes: everything on public subnets (with maybe security groups or network ACLs to restrict it somewhat). Splitting infrastructure into public/private subnets is very tedious, and can at times be difficult to debug. In some of my most recent work it took some time to understand why and how the components needed to be linked to make a combination of an Internet Gateway, Network Firewall, and NAT Gateway work together. Combined with the use of multiple private subnets residing in different AZs meant testing permutations and fussing with the "reachability" analyzer. It is this kind of thing that makes people go with the first thing that works so that they can move on to getting productive work done. Typically one wouldn't serve traffic via a NAT Gateway, and that's true for our case as well. The problem we ran into was not properly configuring VPC endpoints for a workload that heavily interacts with S3 and running up a tiny, but still not ideal, bill for network utilization over the NAT Gateway instead. It's not obvious to may people that interacting with AWS APIs happens over the public Internet even from within AWS infrastructure. The cost of ongoing cost NAT Gateways versus the free per-VPC Internet Gateway feels like it disincentivizes starting with a more secure setup. Ultimately what I'm trying to say is that it's somewhat negligent of AWS to not make an almost universal requirement of modern web and SaaS cloud networking be simple and cheap to implement.