12 ms·
AWS Network Load Balancer
- colmmacc 9y agoIf you're curious to see NLB in action, here's a live demo: http://nlb-34dc3b430638dc3e.elb.us-west-2.amazonaws.com/ http://nlb-34dc3b430638dc3e.elb.us-west-2.amazonaws.com/ , it took about 5 minutes in the console to set it up and no changes on the targets/backends. Massive disclaimer: I work on NLB.
- irl_zebra 9y agoAny idea how long for GovCloud?
- neom 9y agoHello fellow GovCloud user, would love to compare notes if you're game. je@h4x.club :)
- icebraining 9y agoI got to say, asking about GovCloud and with that email address your post sounds like a terrible spear phishing attempt :)
- p0rkbelly 9y agoPretty sure that's the entire point :)
- icebraining 9y agoWas it? neom has posted that email in the past.
- neom 9y agoAin't no dark arts over here. Just a buddy trying to fix cities. <3
- ckozlowski 9y agoDo you have a support POC? I'd reach out to them as they should be able to provide you with a roadmap update. If you're not sure who that is, you can reach out to me at kozlowck at amazon.com.
- knite 9y agoIt sounds like NLB passes through source IP - does that mean outbound flows are through the IGW?
- cperciva 9y agoThere must be some magic happening somewhere, because otherwise outgoing packets would have the wrong source address.
- ec109685 9y agoYou can use direct server return to manipulate the Ethernet frames so that packets don't travel back through the load balancer on the way to the parent switch.
- toast0 9y agoThat generally requires config on the serving hosts, which wasn't mentioned in the setup. I think I saw a reference to adding hosts with a different port number than the service port as well. For people in EC2-VPC (not classic), all their traffic is going through an Amazon NAT anyway, perhaps this new service is setting up translations there. (Note all the references to VPC, and never a mention of EC2-classic)
- bogomipz 9y agoDirect Server Return works at layer 3 not layer 2, its routing and encapsulation - IP in IP, GRE etc.
- mdouglass 9y agoIs there any intent to add TLS termination? That’s a dealbreaker for us switching from the classic load balancer. Otherwise this looks really awesome, thanks!
- nonane 9y agoI don't think they can add TLS termination because of the way it's implemented. NLB runs on Layer 4 - the transport layer where TCP/UDP run on. TLS technically runs on top of the transport layer.
- paulddraper 9y agoHuh? It's no different than ELB.
- mdouglass 9y agoThat’s kind of the answer I was expecting, just hoping it wasn’t the case. From the marketing material they really want you to move, but not having a solution to offload tls makes it impossible for us. And it worries me to see the CLB getting effectively deprecated with it an alternative
- seanp2k2 9y agoALB can term TLS for h2 and wss: https://aws.amazon.com/elasticloadbalancing/ https://aws.amazon.com/elasticloadbalancing/ sounds like that's what you might want?
- mdouglass 9y agoUnfortunately we are currently on a custom TCP-based protocol (we're in the game space). But yes, this is more incentive for us to consider h2 or wss.
- EwanToo 9y agoI'm hopeful AWS will follow this up with ACM supporting SSL certs on instances, so you can run a LetsEncrypt equivalent on each instance, providing TLS end to end encryption
- posnet 9y agoAre there any plans to add UDP support?
- chii 9y agois there a cloudformation template for this demo that i can look at?
- nikolay 9y agoI doubt. Unfortunately, CloudFormations lags severely behind...
- colmmacc 9y agoCloudFormation, CodeDeploy, and ECS all support NLB today :) I used the console to create the simple demo though, so I don't have a template to recreate it. Sorry!
- nikolay 9y agoGood as I'm tired spending days writing Customer Resources!
- sidds 9y agoCloudFormation provides support for Network Load Balancer. Check the 'Type' property mentioned in the CloudFormation templates reference in public documentation http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticloadbalancingv2-loadbalancer.html http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuid...
- friendzis 9y agoDemo page states "Your browser may keep a connection open for a few seconds and re-use it for a reloaded request. If it does, you'll get the same target", but when I attempted to abuse the power of F5, I was alternated between ice cream and bumblebee. If you are going to look at it, attempt time - ~04:50 UTC, remote address from 88.119.128.0/20 network
- 076ae80a-3c97-4 9y agoSame, got a different one each time.
- zob_cloud 9y agoBrowsers typically use a few connections to load a page so that it can load faster. Each of those threads has a different source port, and thus may route to a different target. In Colm's demo, it depends which thread your browser uses when requesting the part including the CSS which decorates the object. In my Chrome on Mac I see 6 TCP connections to the demo NLB tcp4 0 0 192.168.100.101.49615 54.69.111.179.80 ESTABLISHED tcp4 0 0 192.168.100.101.49614 54.69.111.179.80 ESTABLISHED tcp4 0 0 192.168.100.101.49613 54.69.111.179.80 ESTABLISHED tcp4 0 0 192.168.100.101.49612 54.69.111.179.80 ESTABLISHED tcp4 0 0 192.168.100.101.49611 54.69.111.179.80 ESTABLISHED tcp4 0 0 192.168.100.101.49610 54.69.111.179.80 ESTABLISHED Each of those will be routed to the same target, so it's up to your browser to decide which to use for what.
- stelund 9y agoThere is no Security Group for NLB, how is that reasoned?
- jdc0589 9y agothat threw me for a bit of a loop as well. This means that responsibility for doing ACL whitelisting at the edge is now moved from the actual edge, to the security groups on the actual servers responding to request, right? That's do-able and all, but I kind of didn't hate the old paradigm of having an extra layer there.
- colmmacc 9y agoOne way to think of NLB is that it's an Elastic IP address that happens to go to multiple instances or containers, instead of just one. Everything else stays the same.
- stelund 9y agoYeh, it's easy to use it like that for now. I hope they update it later on though. Seems like an missing feature in their otherwise nice firewall rules setup.
- g09980 9y agoThis was discussed a couple of times recently, but answers seemed contradictory. ELB requires pre-warming if you expect sudden high load. But do ALB and NLB? [1] https://news.ycombinator.com/item?id=15085863 https://news.ycombinator.com/item?id=15085863 [2] https://news.ycombinator.com/item?id=14052079 https://news.ycombinator.com/item?id=14052079
- tyingq 9y agoForum post answer, so not 100% official, but it is from an AWS employee. "Yes, while you should see performance increases on ALB in general, for major traffic surges we still recommend a pre-warm" https://forums.aws.amazon.com/message.jspa?messageID=746774 https://forums.aws.amazon.com/message.jspa?messageID=746774
- deanCommie 9y agoI would trust what @collmmac (https://news.ycombinator.com/user?id=colmmacc https://news.ycombinator.com/user?id=colmmacc) is writing above, as he is a Senior Principal on NLB: https://www.linkedin.com/in/colmm1/ https://www.linkedin.com/in/colmm1/
- discodave 9y agoNot sure about ALB, but from the linked blog post for NLB: > Beginning at 1.5 million requests per second, they quickly turned the dial all the way up, reaching over 3 million requests per second and 30 Gbps of aggregate bandwidth before maxing out their test resources.
- uji 9y agoSeems like this limitation has been for long time (since 2009). Curious to know how everyone has been using ELB. To me, ELB seems to be an unfinished product. It must be painful to first predict heavy load on your application and then notify AWS well in advance.
- discodave 9y agoIt's almost like AWS released what they thought was the best product at the time, regretted some of their decisions and then launched other product(s) to replace it.... In this case, it seems like the one-size-fits-all ELB has been replaced by ALB for those using containers, who want L7 LB, and don't need insanity-scale. NLB for those who want massive scale, a dumb pipe and/or need consistent IPs. They could have tried to build these features into ELB but they didn't, they deliberately created new nomenclature to get rid of baggage. Also see: SimpleDB -> DynamoDB. EC2 Classic -> VPCs.
- sgs1370 9y agoThis will motivate me to get everything into VPCs, which I should have done a while ago.
- mooreds 9y agoI love the concept, because not being able to handle TCP traffic was one shortcoming of the new ALB. But that pricing model: Bandwidth – 1 GB per LCU. New Connections – 800 per LCU. Active Connections – 100,000 per LCU. Would be nice to have it added to the simple monthly calculator: https://calculator.s3.amazonaws.com/index.html https://calculator.s3.amazonaws.com/index.html but I had to read the FAQ to find out what those were: https://aws.amazon.com/elasticloadbalancing/faqs/ https://aws.amazon.com/elasticloadbalancing/faqs/
- runako 9y agoIt looks like the deep linking to the LCU page doesn't work (you have to click the tab for Network Load Balancer), so here's what an LCU is from that page: --- An LCU is a new metric for determining how you pay for a Network Load Balancer. An LCU defines the maximum resource consumed in any one of the dimensions (new connections/flows, active connections/flows, and bandwidth) the Network Load Balancer processes your traffic.
- deleted 9y ago[deleted]
- djhworld 9y agoI often feel the simple monthly calculator is largely neglected by AWS. A large number of services are not even featured on the calculator as an option (e.g. Lambda)
- Cieplak 9y agoI wonder what they wrote it in. I'd guess C++, Java or Erlang, or a combination of those.
- Thaxll 9y agoMost likely C/C++, Erlang / Java is not fast enough for this kind of work load.
- tyingq 9y agoIt preserves source ip, so that suggests something like a kernel module, or like Intel's DPDK. Either (practically) rules out anything but C, doesn't it?
- Thaxll 9y agoOr they could just encapsulate the packet to preserve the IP...
- tyingq 9y agoHappy to be educated. There's some easy way to alter the destination IP that way?
- Thaxll 9y agoLike mention above IPVS: http://www.linuxvirtualserver.org/VS-IPTunneling.html http://www.linuxvirtualserver.org/VS-IPTunneling.html This is heavily used by Facebook for their loadblancer on their racks.
- tyingq 9y agoAh, ok. A kernel module, as I mentioned :)
- omribahumi 9y agoThere was a talk about this in SREcon Europe 2015: https://www.usenix.org/conference/srecon15europe/program/presentation/shuff https://www.usenix.org/conference/srecon15europe/program/pre...
- irl_zebra 9y agoIt's not showing up in GovCloud. This stuff always takes longer for that.
- Thaxll 9y agoWhen is http2 coming to ELB ... :/ It should be their #1 priority.
- colmmacc 9y agoELB's Application Load Balancer (ALB) supports HTTP/2 termination. More in the original launch announcement: https://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/ https://aws.amazon.com/blogs/aws/new-aws-application-load-ba... and https://aws.amazon.com/elasticloadbalancing/details/#details https://aws.amazon.com/elasticloadbalancing/details/#details
- Thaxll 9y agohttp2 termination is not very useful, most people want ELB -> backend instances.
- seanp2k2 9y agoin case anyone is confused about this sub-thread: """ Application Load Balancers provide native support for HTTP/2 with HTTPS listeners. You can send up to 128 requests in parallel using one HTTP/2 connection. The load balancer converts these to individual HTTP/1.1 requests and distributes them across the healthy targets in the target group using the round robin routing algorithm. Because HTTP/2 uses front-end connections more efficiently, you might notice fewer connections between clients and the load balancer. Note that you can't use the server-push feature of HTTP/2. """ ( http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#listener-configuration http://docs.aws.amazon.com/elasticloadbalancing/latest/appli... )
- acejam 9y agoYes it is. For the client, multiplexing multiple requests across a single connection and only needing to negotiate TLS once can have huge benefits, depending on the page/site. You are after all building your app/site for your end users, right?
- tjholowaychuk 9y agoAwesome! I have the perfect project for this haha. Does it still work with ECS integration?
- Thaxll 9y agoYes it does ( target group integration )
- niksan852 9y agoHello everyone , We launched new Carding and Hacking forum www.cardvilla.net which is 100% Free to join and you can get free credit cards, paypal, SMTP, RDP, Webmail, Mailer, Porn account, Dumps, Hacking and Carding Tutorials, and we are also doing Western Union money tranfer and Electronics Shipment and much more you cant even imagine so join now for free and enjoy all free stuffs. . www.cardvilla.net
- dfischer 9y agoThis is perfect. Need something like this to load balance A records for dynamic domains off apex. We can most likely use the static IP address perfectly for this. Also did I read it wrong or this is actually cheaper than ALB? Awesome! Can't wait to dig in more!
- davidbrownct 9y agoYes, NLB is priced the same as ALB hourly but 25% cheaper on bandwidth (LCUs).
- dlhavema 9y agoany idea if you can use this behind API gateway if it is not itself public?
- muddley 9y agoyes of course. The NLB is public and API GW sits on the edge locations
- paulddraper 9y agoStatic IP, source IP, and zonality are game changing. Unfortunately, it lacks a very significant existing feature of ELB: SSL/TLS termination. It's very convient to manage the certs in AWS without having to deploy them to dedicated EC2 instances.
- 9point6 9y agoIt won't ever be possible to do this as the NLB runs a few network layers below where TLS runs
- paulddraper 9y agoDoes it? It has HTTPS health checks.
- 9point6 9y agoI expect the health-checking happens independently of the routing. The routing will just act upon a list of routes which is modified independently by the health check.
- jacobr1 9y agoThis is what their ALB service if for.
- hexsprite 9y agoso would a websocket-based application be better off using NLB?
- zob_cloud 9y agoBetter off depends on what your workload goals are. If you want path or host name based routing, Application load balancer may be a better fit as it natively supports WebSockets. If your goal is long lived sessions (weeks and months, not minutes and hours), Network load balancer is probably a better fit.
- seanp2k2 9y agoALBs are great for WS and can term SSL for WSS. Just use "HTTP" and "HTTPS" protos on the target groups and it'll work. It's a bit confusing but it works.
- inertial 9y agoFeature request : Please allow weighted load balancing i.e. ability to distribute traffic in a user specified ratio (weights) to different sized instances.
- colmmacc 9y agoFor now here's a work-around that I use: create multiple listeners/ports on the larger instances and add them as targets. Containers is a great approach here too; load up the bigger instances with more containers and register each container as the targets.
- bashtoni 9y agoBetter: distribute to the instance with the lowest % CPU by default, ala the Google Cloud NLB.
- jedberg 9y agoThat's very hard to get right. If you don't have exactly the right cool downs and back offs, you get oscillating load.
- colmmacc 9y agoThis is what ELB Classic did for a long time - but we're experimenting with new algorithms. The problem with this approach is that it's not cache friendly. When you bring in a new server to a busy workload, it gets all of the new connections. That can put a lot of pressure on that host - and if the application relies on caching, which most do in some form, it can really make performance terrible and even trigger cascading failures. Another problem is that if a single host is broken or misconfigured, throwing 500 errors, it is often also the fastest and lowest CPU box, because that isn't very expensive. It can suck in and blackhole all of the traffic. Based on how these issues work out at scale, we've moved beyond simple load based load balancing (I know that sounds counter-intuitive) and into algorithms that try to achieve a better balance for a wider range of scenarios.
- syncerr 9y agoSeems to remarkably decrease latency (380ms -> 109ms). Running some tests: # ab -n 400 http://nlb-34dc3b430638dc3e.elb.us-west-2.amazonaws.com/ Time per request: 108.779 [ms] (mean, across all concurrent requests) # ab -n 400 <public server via ELB> 381.933 # ab -n 400 <public server via ALB> 380.632 # (for reference) ab -n 400 https://www.google.com/ 190.536 # (for reference) ab -n 400 https://sandbox-api.uber.com/health/ 107.680 If you're wiling to terminate SSL, this looks like it could be a solid improvement.
- geocar 9y agoI've seen a similar improvement using the EnableProxyProtocol policy, which required a bit of code: Time per request: 88.400 [ms] (mean, across all concurrent requests) versus public server via the regular HTTP proxy: Time per request: 415.859 [ms] (mean, across all concurrent requests) For reference: $ ab -n 400 https://www.google.com/ Time per request: 168.438 [ms] (mean, across all concurrent requests)
- shampster 9y agoI was so excited before I realized this wouldn't terminate http(s) traffic. An IP anycast based load balancer ALB would be nice.
- deafcalculus 9y agoHow does failover across zones work? The blog post says there's one static ip per zone. I suppose www.mydomain should have multiple A records each pointing to an elastic ip in a zone. What happens when one zone entirely fails? Does it need a DNS change at this point? Or does the NLB have a different IP with which it can do BGP failover?
- felipelemos 9y agoThis is a job for a CDN
- krallin 9y agoAWS provides you with a number of DNS records for each NLB: - One record per zone (which maps to the EIP for that zone) - A top-level record that includes all active zones (these are all zones you have registered targets in, IIRC) The latter record is health checked, so if an AZ goes down, it'll stop advertising it automatically (there will be latency of course, so you'll have some clients connecting to a dead IP, but if we're talking unplanned AZ failure, that's sort of expected). That said, this does mean you probably shouldn't advertise the IPs directly if you can avoid it, yes. (disclaimer: we evaluated NLB during their beta, so some of this information might be slightly outdated / inaccurate)
- deafcalculus 9y agoWon't DNS failover be painfully slow? Some clients ignore small TTL values. I've seen DNS updates taking several hours to propagate. I thought one of the advantages of multiple zones is that zonal failover can happen with "zero" downtime (this seems to be the case with Amazon RDS).
- colmmacc 9y agoThe default answer includes multiple A records, so if clients can't reach one of the IPs, they try another. There's no need for anything to propagate for that to kick in, it's just ordinary client retry behavior. We do also withdraw an IP from DNS if it fails; when we measure it, we see that over 99% of clients and resolvers do honor TTLs and the change is effected very quickly. We've been using this same process for www.amazon.com for a long time. Contrast to an alternative like BGP anycast, where it can take minutes for an update to propagate as BGP peers share it with each other in sequence.
- kadiyala 9y agoI was just wondering if this is something purely developed inside amazon or is it backed by an ADC like NetScaler or F5. does anyone know any detail ? I'm assuming that classic load balancer is some third-party or old framework and this is something amazon developed internally.
- Corrado 9y agoI would assume that it's something developed internally at Amazon. Networking inside of AWS isn't standard fare and I doubt something like NetScaler or F5 products would be able to be used. Generally speaking, they aren't using TCP/IP behind the curtain, to move packets between nodes. AWS has even created their own routing hardware/software because no other company could do what they need at the scale that they need. See this video for more information: https://www.youtube.com/watch?v=St3SE4LWhKo https://www.youtube.com/watch?v=St3SE4LWhKo
- discodave 9y agoAWS & Amazon uses a LOT of routers from one of these vendors. Although, they would probably try and avoid baking it into a public-facing product like this.
- ckozlowski 9y agoNot necessarily: https://www.geekwire.com/2017/amazon-web-services-secret-weapon-custom-made-hardware-network/ https://www.geekwire.com/2017/amazon-web-services-secret-wea...
- GeneticGenesis 9y agoNo chance that I'll jump into another new load balancer product from Amazon any time soon. ALB has significant deficiencies that AWS don't warn you about, and you only find then at tens of thousands of RPS. Still waiting on that fix, AWS.
- newhere420 9y agoIf they won't warn us, could you please warn us? - Fellow ALB user.
- GeneticGenesis 9y agoSure. Whenever a "config change" (Note: this includes adding or removing targets to a target group, EG Autoscaling) happens on an ALB, the ALB drops all active connections, and re-establishes them at once, at high load, this obviously causes significant load spikes on any underlying service. You can see this happening by looking at the "Active Connection Count" graphs from your ALB, and adding or removing an instance from an ASG. At 30+GBPS and over 20kRPS, removing one instance can cause absolute chaos.
- la6470 9y agoAnd how often you do confit changes?
- colmmacc 9y agoWow that sounds awful - but thankfully this isn't typical. I'm going to go digging for a case-id/issue and see what's going on myself (please e-mail the case if you have one). Re-configurations are routine and graceful. From your description it may be that you have long lived connections that build up over time, at a rate that targets can easily handle, but that the re-connect spikes associated with a target failure/withdrawal are too intense. This is a challenge I've seen with web sockets: imagine building up 100,000 mostly-idle web sockets slowly over time, even a modest pair of backends can handle this. But then a backend fails, and 50,000 connections come storming in at once! Another scenario is adding an "idle" target to a busy workload, but it not being able to handle the increased rate of new connections it will get. Software that relies on caching (including things like internal object caches) often can handle a slow ramp-up, but not a sudden rush. We're currently experimenting with algorithms that allow customers to more slowly ramp-up the incoming rate of connections in these kinds of scenarios. Anyway, those are guesses, so I may be wrong about your case, but hopefully the information is still useful to others reading.
- sigi45 9y agonice. Especially the ip thing.
- baccredited 9y agoDoes it support ipv6?
- gregmac 9y agoI have just finished setting up a new front-end for a few services (we are just about to start migrating production systems to it). I was aiming to use static IPs (for client firewall rules), and simplify networking configuration, so what I ended up with is an auto-scaling group of HAProxy systems that run a script every couple of minutes to assign themselves an elastic IP from a provided list. Route 53 is configured with health checks to only return the IP(s) that are working. The HAProxy instances also continuously read their target auto-scaling groups to update backend config, and do ssl terminating, also running the Let's Encrypt client. Most services are routed by host name, but a couple older ones are path-based and there are some 301 redirects. I think NLB could replace the elastic IP and route53 part of this setup, but I'd still need to do SSL, routing, and backends. It's too bad, because my setup is one that could be used nearly anywhere that has more than one public-facing service, but there's not much built-in to help - I had to write quite a few scripts to get everything I needed.
- gnur 9y agoHave you tried/evaluated traefik? It sounds like it could do nearly everything you just mentioned.
- mark242 9y agoI don't understand the pricing model. 800 new connections per hour, for $0.006? Isn't that extremely expensive? 80,000 connections for $0.60 in an hour is $432 per month for not a whole lot of traffic. edit: Okay, it's 800 new connections per second, per the ELB pricing page, under "LCU details". The cost for 80k connections in an hour is effectively constrained by the bandwidth, eg if there's very low bandwidth it's $0.006/hour or $4.32/month.