6 ms·

This was discussed a couple of times recently, but answers seemed contradictory. ELB requires pre-warming if you expect sudden high load. But do ALB and NLB? [

by g09980 9y ago

This was discussed a couple of times recently, but answers seemed contradictory. ELB requires pre-warming if you expect sudden high load. But do ALB and NLB? [1] https://news.ycombinator.com/item?id=15085863 https://news.ycombinator.com/item?id=15085863 [2] https://news.ycombinator.com/item?id=14052079 https://news.ycombinator.com/item?id=14052079

tyingq 9y ago
Forum post answer, so not 100% official, but it is from an AWS employee. "Yes, while you should see performance increases on ALB in general, for major traffic surges we still recommend a pre-warm" https://forums.aws.amazon.com/message.jspa?messageID=746774 https://forums.aws.amazon.com/message.jspa?messageID=746774
- deanCommie 9y ago
  I would trust what @collmmac (https://news.ycombinator.com/user?id=colmmacc https://news.ycombinator.com/user?id=colmmacc) is writing above, as he is a Senior Principal on NLB: https://www.linkedin.com/in/colmm1/ https://www.linkedin.com/in/colmm1/
discodave 9y ago
Not sure about ALB, but from the linked blog post for NLB: > Beginning at 1.5 million requests per second, they quickly turned the dial all the way up, reaching over 3 million requests per second and 30 Gbps of aggregate bandwidth before maxing out their test resources.
uji 9y ago
Seems like this limitation has been for long time (since 2009). Curious to know how everyone has been using ELB. To me, ELB seems to be an unfinished product. It must be painful to first predict heavy load on your application and then notify AWS well in advance.
- discodave 9y ago
  It's almost like AWS released what they thought was the best product at the time, regretted some of their decisions and then launched other product(s) to replace it.... In this case, it seems like the one-size-fits-all ELB has been replaced by ALB for those using containers, who want L7 LB, and don't need insanity-scale. NLB for those who want massive scale, a dumb pipe and/or need consistent IPs. They could have tried to build these features into ELB but they didn't, they deliberately created new nomenclature to get rid of baggage. Also see: SimpleDB -> DynamoDB. EC2 Classic -> VPCs.
  - chronid 9y ago
    good > perfect is somewhat standard thinking in AWS.
colmmacc 9y ago
Each NLB starts out with several gigabits of capacity per availability zone, and it scales horizontally from there (theoretically to Terabits). That's more capacity than many of the busiest web-sites and web-services in the world need. If you expect an instantaneous load of more than about 5Gbit/sec, in those situations we work directly with customers via AWS Support. We really try to understand the load, make sure that the right mechanisms are in place. At that scale, our internal DDOS mitigation systems also come into play. (It's not a constraint of NLB). The load test in the blog post was done with an NLB, and was done with no pre-provisioning or pre-warming and allowed us to get to 3M RPS and 30Gbit/sec, which is when we exhausted the capacity of our test backends. ALBs start out with less capacity, and are constrained more by requests than bandwidth. I don't have a precise number because it depends on how many rules you have configured and which TLS ciphers are negotiated by your clients, but the numbers are high enough that customers routinely use ALBs to handle real-world spiky workloads, including supporting Super Bowl ads and flash sales. Each ALB can scale into the tens of gigabits/sec before needing to shard. ALB also has a neat trick up its sleeve: if you add backends, we scale up, even if there's no traffic. We assume the backends are there to handle expected load. So in that case it has "more" capacity than the backends behind it. That goes a long way to avoiding some of the scaling issues that impacted ELB early in its history. If you have a workload that you're worried about, feel free to reach out to me and we'll be happy to work with you. colm _AT_ amazon.com.
- vacri 9y ago
  > [upgrading ELB], in those situations we work directly with customers via AWS Support. This is painful, though. I don't know about Sep 2017, but in 2016 upscaling your ELB involved answering a block of 21 or so questions in a list only given to you after you engage with support, which had some pretty esoteric items on it. It was decidedly un-AWS-ey.
  - colmmacc 9y ago
    It was definitely un-AWS-ey, and there's an almost visceral pain response on our faces when we don't have a self-service API for something. This is improving all of the time and I think is already much better, with some big specific improvements ... I'll do my best to share what I can here. First things first, with NLB our experience is that pre-warming is never necessary. Each NLB starts out with a huge volume of capacity, beyond the needs of even the largest systems we support, and each NLB can theoretically scale to terabits of traffic. Our first big improvement for ALB was that the basic scalability and performance of ALB is at a point that in all but a very small number of cases (think of some of the busiest services in the world), customers don't need to do anything. This is the pay-off from a lot of hard work focused on low-level performance. Our second big improvement, for both ALB and Classic ELBs, was a mix of more generous capacity buffers, and pro-active and responsive scaling systems. Together, these mean that we can race ahead of our customers load requirements. Another item that's helped is that for the truly big scaling cases, which is DDOS preparedness, we now have the AWS Shield service to manage that process in consultation with the customer. That's useful if your needs are more nuanced and custom than the DDOS protection that is included with ELB by default. This gets into things such as how your application is configured to handle the load. With all of these improvements, ALB does not require pre-warming for the vast majority of real-world workloads. However, after years of pre-warming as a thing, we have customers who have incorporated it into their operational workflows, or who rely on it for extra peace of mind. We do want to continue to support that for our customers.

by g09980 9y ago

tyingq 9y ago

Forum post answer, so not 100% official, but it is from an AWS employee. "Yes, while you should see performance increases on ALB in general, for major traffic surges we still recommend a pre-warm" https://forums.aws.amazon.com/message.jspa?messageID=746774 https://forums.aws.amazon.com/message.jspa?messageID=746774

deanCommie 9y ago
I would trust what @collmmac (https://news.ycombinator.com/user?id=colmmacc https://news.ycombinator.com/user?id=colmmacc) is writing above, as he is a Senior Principal on NLB: https://www.linkedin.com/in/colmm1/ https://www.linkedin.com/in/colmm1/

discodave 9y ago

Not sure about ALB, but from the linked blog post for NLB: > Beginning at 1.5 million requests per second, they quickly turned the dial all the way up, reaching over 3 million requests per second and 30 Gbps of aggregate bandwidth before maxing out their test resources.

uji 9y ago

Seems like this limitation has been for long time (since 2009). Curious to know how everyone has been using ELB. To me, ELB seems to be an unfinished product. It must be painful to first predict heavy load on your application and then notify AWS well in advance.

discodave 9y ago
It's almost like AWS released what they thought was the best product at the time, regretted some of their decisions and then launched other product(s) to replace it.... In this case, it seems like the one-size-fits-all ELB has been replaced by ALB for those using containers, who want L7 LB, and don't need insanity-scale. NLB for those who want massive scale, a dumb pipe and/or need consistent IPs. They could have tried to build these features into ELB but they didn't, they deliberately created new nomenclature to get rid of baggage. Also see: SimpleDB -> DynamoDB. EC2 Classic -> VPCs.
- chronid 9y ago
  good > perfect is somewhat standard thinking in AWS.

colmmacc 9y ago

Each NLB starts out with several gigabits of capacity per availability zone, and it scales horizontally from there (theoretically to Terabits). That's more capacity than many of the busiest web-sites and web-services in the world need. If you expect an instantaneous load of more than about 5Gbit/sec, in those situations we work directly with customers via AWS Support. We really try to understand the load, make sure that the right mechanisms are in place. At that scale, our internal DDOS mitigation systems also come into play. (It's not a constraint of NLB). The load test in the blog post was done with an NLB, and was done with no pre-provisioning or pre-warming and allowed us to get to 3M RPS and 30Gbit/sec, which is when we exhausted the capacity of our test backends. ALBs start out with less capacity, and are constrained more by requests than bandwidth. I don't have a precise number because it depends on how many rules you have configured and which TLS ciphers are negotiated by your clients, but the numbers are high enough that customers routinely use ALBs to handle real-world spiky workloads, including supporting Super Bowl ads and flash sales. Each ALB can scale into the tens of gigabits/sec before needing to shard. ALB also has a neat trick up its sleeve: if you add backends, we scale up, even if there's no traffic. We assume the backends are there to handle expected load. So in that case it has "more" capacity than the backends behind it. That goes a long way to avoiding some of the scaling issues that impacted ELB early in its history. If you have a workload that you're worried about, feel free to reach out to me and we'll be happy to work with you. colm _AT_ amazon.com.

vacri 9y ago
> [upgrading ELB], in those situations we work directly with customers via AWS Support. This is painful, though. I don't know about Sep 2017, but in 2016 upscaling your ELB involved answering a block of 21 or so questions in a list only given to you after you engage with support, which had some pretty esoteric items on it. It was decidedly un-AWS-ey.
- colmmacc 9y ago
  It was definitely un-AWS-ey, and there's an almost visceral pain response on our faces when we don't have a self-service API for something. This is improving all of the time and I think is already much better, with some big specific improvements ... I'll do my best to share what I can here. First things first, with NLB our experience is that pre-warming is never necessary. Each NLB starts out with a huge volume of capacity, beyond the needs of even the largest systems we support, and each NLB can theoretically scale to terabits of traffic. Our first big improvement for ALB was that the basic scalability and performance of ALB is at a point that in all but a very small number of cases (think of some of the busiest services in the world), customers don't need to do anything. This is the pay-off from a lot of hard work focused on low-level performance. Our second big improvement, for both ALB and Classic ELBs, was a mix of more generous capacity buffers, and pro-active and responsive scaling systems. Together, these mean that we can race ahead of our customers load requirements. Another item that's helped is that for the truly big scaling cases, which is DDOS preparedness, we now have the AWS Shield service to manage that process in consultation with the customer. That's useful if your needs are more nuanced and custom than the DDOS protection that is included with ELB by default. This gets into things such as how your application is configured to handle the load. With all of these improvements, ALB does not require pre-warming for the vast majority of real-world workloads. However, after years of pre-warming as a thing, we have customers who have incorporated it into their operational workflows, or who rely on it for extra peace of mind. We do want to continue to support that for our customers.