4 ms·

Has anyone heard of implementing an architecture where you have normal server backend architecture, while using serverless to handle traffic spikes and any fail

Has anyone heard of implementing an architecture where you have normal server backend architecture, while using serverless to handle traffic spikes and any failover? If you properly modularize your backend codebase, it shouldn't be excessively costly to duplicate it across a serverless architecture and in some sort of node instance. My biggest issue with serverless is the high cost at scale, which could be mitigated with this type of architecture. Are there any other benefits or concerns regarding something like this?

liveoneggs 8y ago
why would you do this when regular autoscale could just be used instead?
- wahnfrieden 8y ago
  Autoscale doesn't allow you to scale to zero (as any requests would fail, rather than wait on resources to scale up). EC2 autoscaling takes a minute or two. This means you need to run with enough headroom to absorb spikes before you're able to spin up more resources. Depending on traffic patterns this can get very expensive (or not a problem at all, with predictable loads).
  - liveoneggs 8y ago
    if you're worried about the "zero" traffic situation I am not sure you also need to worry much about the "massive" traffic situation
    - wahnfrieden 8y ago
      I am sure :) Our customers are concentrated around N. America, with minimal activity at night and weekends. We have a complex product with a wide surface area. Some of our services don't see usage constantly throughout the day. Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling. Then there's staging environments...
      - liveoneggs 8y ago
        The smallest ec2 instance is $4/m on-demand (or $8/m for a usable amount of memory)
        wahnfrieden 8y ago
        >Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling. Smallest EC2 instance will fall over with a sudden traffic spike, and requests will fail for the couple minutes it takes for autoscaling to complete. Additionally for HA, a machine must be able to die without exposing us to the risk of a traffic spike taking us down.
    - larrywright 8y ago
      Actual use case I heard of: Outage reporting system for a utility. You know, the page you go look at when your power goes out. The majority of the time, this page receives no traffic. When there’s an outage, the traffic spikes immensely. When the outage is over, the traffic drops back to near-zero.
      - gregmac 8y ago
        An interesting way to work around this would be to have the ability to scale with mixed instance types. For example, your base application runs with t2.micro. As soon as traffic starts going up, you launch additional medium/large instances. The load balancer would have to understand this also, and do weighted routing (ideally based on app server loads) so the first micro instance doesn't handle the same volume of traffic as the rest. I've never seen a setup like this before, and frankly, part of me wonders whether the savings would be worth the effort of building it in the first place (vs just running with medium instances, or just scaling to dozens of micros to handle the load, or just using serverless).
      - manigandham 8y ago
        All you need is a CDN. Cheap, easy, and decades of reliability. You don't need infrastructure for this.
    - Touche 8y ago
      Every company has parts of their system that has "zero" traffic from time to time.
toomuchtodo 8y ago
Sounds more expensive than scaling up more servers on demand.
- brazzledazzle 8y ago
  What I hear a lot is that the scaling can take too long. I wonder if a hybrid where you use lambda as a temporary buffer would be the most cost effective.
  - deleted 8y ago
    [deleted]
  - pageald 8y ago
    In my experience using AWS, you can spin up new autoscaled instances pretty quickly. I don't even have a custom AMI, I use a generic image and run startup scripts to kick off my server software. Takes < 3 minutes to spin up a new one and start accepting requests. If that's not fast enough, you can bake an AMI and scale more proactively (e.g. scale when your server is 70% utilized rather than 95%)
aglikson 8y ago
Actually we have started doing some initial exploration of such an architecture. But don't have results we can share yet. Maybe in one of the follow-on blog posts :-) I agree that it could make a lot of sense in some cases.
- bsbechtel 8y ago
  I would be interested in hearing how it goes :) Do I follow the OP, or subscribe somewhere else to get notified when you release it?

liveoneggs 8y ago

why would you do this when regular autoscale could just be used instead?

wahnfrieden 8y ago
Autoscale doesn't allow you to scale to zero (as any requests would fail, rather than wait on resources to scale up). EC2 autoscaling takes a minute or two. This means you need to run with enough headroom to absorb spikes before you're able to spin up more resources. Depending on traffic patterns this can get very expensive (or not a problem at all, with predictable loads).
- liveoneggs 8y ago
  if you're worried about the "zero" traffic situation I am not sure you also need to worry much about the "massive" traffic situation
  - wahnfrieden 8y ago
    I am sure :) Our customers are concentrated around N. America, with minimal activity at night and weekends. We have a complex product with a wide surface area. Some of our services don't see usage constantly throughout the day. Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling. Then there's staging environments...
    - liveoneggs 8y ago
      The smallest ec2 instance is $4/m on-demand (or $8/m for a usable amount of memory)
      - wahnfrieden 8y ago
        >Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling. Smallest EC2 instance will fall over with a sudden traffic spike, and requests will fail for the couple minutes it takes for autoscaling to complete. Additionally for HA, a machine must be able to die without exposing us to the risk of a traffic spike taking us down.
  - larrywright 8y ago
    Actual use case I heard of: Outage reporting system for a utility. You know, the page you go look at when your power goes out. The majority of the time, this page receives no traffic. When there’s an outage, the traffic spikes immensely. When the outage is over, the traffic drops back to near-zero.
    - gregmac 8y ago
      An interesting way to work around this would be to have the ability to scale with mixed instance types. For example, your base application runs with t2.micro. As soon as traffic starts going up, you launch additional medium/large instances. The load balancer would have to understand this also, and do weighted routing (ideally based on app server loads) so the first micro instance doesn't handle the same volume of traffic as the rest. I've never seen a setup like this before, and frankly, part of me wonders whether the savings would be worth the effort of building it in the first place (vs just running with medium instances, or just scaling to dozens of micros to handle the load, or just using serverless).
    - manigandham 8y ago
      All you need is a CDN. Cheap, easy, and decades of reliability. You don't need infrastructure for this.
  - Touche 8y ago
    Every company has parts of their system that has "zero" traffic from time to time.

toomuchtodo 8y ago

Sounds more expensive than scaling up more servers on demand.

brazzledazzle 8y ago
What I hear a lot is that the scaling can take too long. I wonder if a hybrid where you use lambda as a temporary buffer would be the most cost effective.
- deleted 8y ago
  [deleted]
- pageald 8y ago
  In my experience using AWS, you can spin up new autoscaled instances pretty quickly. I don't even have a custom AMI, I use a generic image and run startup scripts to kick off my server software. Takes < 3 minutes to spin up a new one and start accepting requests. If that's not fast enough, you can bake an AMI and scale more proactively (e.g. scale when your server is 70% utilized rather than 95%)

aglikson 8y ago

Actually we have started doing some initial exploration of such an architecture. But don't have results we can share yet. Maybe in one of the follow-on blog posts :-) I agree that it could make a lot of sense in some cases.

bsbechtel 8y ago
I would be interested in hearing how it goes :) Do I follow the OP, or subscribe somewhere else to get notified when you release it?