6 ms·
Switch to VPC Endpoints from Nat Gateways to Reduce Bandwidth Charges
- worldofmatthew 4y agoOr don't use AWS as other providers provide multiple terabytes of bandwidth for a couple of dollars per month.
- victor106 4y agoWhich providers are these?
- hhw 4y agoPretty much any colo or dedicated/bare metal provider. $0.07 to $0.12 per Mb is the going rate for most carriers at any appreciable volume, and even the higher end carriers are less than 3x that. To be fair, big tech despite their massive volume pay much higher rates than small networks because the carriers charge them enough to fully cover their costs to build out their networKs, while they make all their profits from selling their excess capacity to the little guys for pennies on the dollar.
- hermitdev 4y agoAt $0.07/MB, your talking over $70K to transfer a TB. Am I missing something here? Because I'd say $70k is quite a bit more than "a couple of dollars" the GP mentions. (No, I dont deal with cloud pricing at all, so I'm a little perplexed here)
- hhw 4y agoNetwork at the wholesale level is always measured in Mbps using 95th percentile, not in data transferred (average sustained, equivalent to 50th percentile because we're talking about 5 minute samples of interface counters over the course of a month). Note I used a small b in Mb. Depending on the variability of traffic patterns, that usually works out to be on average ~200GB* transferred per Mbps of 95th percentile over a period of a month. Meaning a TB would work out to about $0.35. *A long, long time ago, I looked at about 1000 co-location customers' MRTG stats and compared their monthly 95th percentile Mbps to their average sustained data transfer in GB, and something like 90% of them were between 150GB-250GB per Mb and 98% of them were between 180GB-220GB. Many people assume 324GB which would require their traffic to be perfectly flatlined throughout the month, which obviously rarely ever happens.
- davidro80 4y ago
- Kudos 4y agoThey're talking about throughput not transfer. Like a 100Mbps link, not 100MB data transferred total.
- WatchDog 4y agoMisleading title, VPC endpoints aren’t a replacement for NAT gateways. Traffic destined for certain AWS services can be offloaded to a VPC endpoint, when it otherwise may have been routed via your NAT gateway. However you still need something to do NAT if you are running an ipv4 private network and need to access the internet
- deleted 4y ago[deleted]
- sigstoat 4y agoas you note they’re only available for some services, and sometimes only partially for a service. check the docs very carefully before getting your hopes up about dropping the bar gateways.
- nine_k 4y agoYou may not need a NAT proper if all you expose / do is HTTP requests. Run a proxy / LB on an instance that has a public IPv4 and a VPC network interface. Don't NAT or route between these networks, just let the proxy listen on the public interface, and contact the backend servers on the private interface.
- andrewguenther 4y agoNAT Gateways also support bandwidth, availability, and performance properties that most customers just don't need. AWS networking services which charge by GB processed are always a cost disaster waiting to happen. I built my own NAT AMI[1] that costs <$4/month and has zero bandwidth charges and is good enough for most people. [1]: https://github.com/AndrewGuenther/fck-nat/ https://github.com/AndrewGuenther/fck-nat/
- aynsof 4y agoMy first question was answered in the readme: "But what about AWS' NAT Instance AMI?" The official AWS supported NAT Instance AMI hasn't been updates since 2018, is still running Amazon Linux 1 which is now EOL, and has no ARM support, meaning it can't be deployed on EC2's most cost effective instance types. fck-nat. I had no idea this was the case. Thank you for making this!
- deleted 4y ago[deleted]
- barefeg 4y agoIs it possible to do this in EKS?
- calgoo 4y agoThere is a aws doc on their page somewhere where they explain how to setup eks on private vpc subnets and using the endpoints to to access the needed aws services. Edit, link: https://docs.aws.amazon.com/eks/latest/userguide/private-clusters.html https://docs.aws.amazon.com/eks/latest/userguide/private-clu...
- abofh 4y agoStrictly speaking yes, but there are limitations depending on what you do - you can't, for example, do route53 or ACM API's via private link (private-CA does, but not the general purpose CA) - so things like external-dns and the ACK ALB+Certificate configuration can't work via auto discovery without NAT or being on a public instance. But EKS by itself, you can absolutely do with private subnets and private link API's.
- jpgvm 4y agoIf you need a generic replacement for NAT Gateway you can build one using Transit Gateway and a pair of router VMs running GRE+BGP. Amazon has most of an example here using quagga (but CloudFormation, ick): https://github.com/aws-samples/aws-transit-gateway-connect-sample https://github.com/aws-samples/aws-transit-gateway-connect-s...
- datalopers 4y agoJust allocate your web crawlers a public IPs and use an internet gateway. Such a weird self-enforced limitation they’ve envisioned, not clear why they think there’s no choice but NAT
- busterarm 4y agoWhy make your web crawlers reachable via the internet? You're one misconfigured security group away from your shit being owned.
- datalopers 4y agoI mean you said it yourself.. that's why you have security groups? You're either paying a ton to use NAT gateway, or setting up a dedicated box to act as your in-house NAT, or configuring security groups. It's pretty simple: no inbound connections except from the VPC. Also even if there's no firewall at all, how does that mean your machines are getting owned? My boxes listen on precisely one port: a heavily locked down sshd (which isn't listening on that interface anyway)
- fuzzybear3965 4y agoI don't understand > listen on one port ... which isn't listening on that interface anyway Would you mind please elaborating, here? And which interface does sshd not listen on?
- datalopers 4y agosshd is only listening on port 22 on the private IP (the VPC) not the public IP of each machine. I then connect into my VPC through a bastion host running wireguard.
- terom 4y agoThat alone will not prevent connections to port 22 on the public IP: the 1:1 non-port-based NAT means that any incoming packets to the public IP will show up at your instance with the private IP as their destination adderess. The TCP/IP stack on your instance knows nothing about the public IPv4 address.
- sigstoat 4y agoso i keep having people froth at the mouth about how it is more secure to use VPC endpoints to communicate to AWS services. is there any practical mechanism by which not using the VPC endpoints can be insecure, which does not also affect VPC endpoints? i usually get some hand waving about man in the middle attacks, but i figure if you can insert that into an AWS data center you can probably just see all the VPC internal traffic anyways. and one should be using TLS to talk to the services anyways.
- spydum 4y agoIf you don't use vpc endpoints, usually you end up having to open network traffic to all of a particular aws service, not just yours. This means as an attacker, I can also go spin up (s3/lambda/etc) and if your app was vulnerable in some way, I could exfil data to those aws services (or leverage them in RFI/CSRF/other attacks) in a way that shouldn't be possible.
- sigstoat 4y ago> you end up having to open network traffic to all of a particular aws service, not just yours > I could exfil data to those aws services I don't think any of the VPC endpoints have any restrictions on them; that is, once I've made an S3 endpoint in my VPC, it can be used to access any bucket. So... it seems like it could be used just as easily to exfiltrate data?
- cldellow 4y agoI haven't done it, but the docs seem to imply that some VPC endpoints, including S3, can support the usual policy documents to describe which actions/principals/resources are permitted: https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html#edit-vpc-endpoint-policy-s3 https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpo...
- different_sort 4y agoThat is indeed the case.
- x3n0ph3n3 4y agoVPC Endpoints are not free, and if you use enough different AWS services, the cost starts to really go up.
- solatic 4y agoAnother way to reduce NAT gateway charges is to simply not need them in your network architecture: IPv6-only, immutable infrastructure (i.e. pulling updates while building the AMI in a different environment so that there is no need to pull from the Internet in production)...
- miyuru 4y agoyou don't need to go IPv6 only, just adding dual-stack IPv6 with egress only gateway can reduce bandwidth costs.