8 ms·

AWS Lambda Web Adapter

134 points by cebert 2y ago

Thaxll 2y ago
Someone with experiance with Lambda, how does connection pooling works, are tcp connection re-used at all?
- remram 2y ago
  There's no direct connection to the Lambda container, it goes through a load balancer. The load balancer can keep connections open for reuse.
- zyl1n 2y ago
  As you might have guessed, connection pools don't work well with Lambda. For database connection pool, they came up with another chargeable service, RDS Proxy, to fix the problem created by Lambda :)
- tiew9Vii 2y ago
  A lambda lives for a relatively short period of time, ephemeral. It only handles a single request at a time but will live long enough to serve multiple requests. If another request comes in while one is in process a new lambda can spin up. As a lambda lives for a period of time you can create a pool for that individual lambda. That pool lives for the life of the lambda. Now for things like HTTP clients this is generally OK. If you have a Postgres pool etc, it's not ok. Postgress doesn't like lots of connections and you will have POOL_SIZE * NUMBER_OF_ACTIVE_LAMBDA connections. So unless you limit the number of Lambda's you have using reserved capacity Lambda's are simply not a good fit for running a PG pool locally inside the lambda for web stuff. If you have a batch job, that runs once a day, with a single lambda, inserts a bunch of records in to postgres or similar, then a small pool in the lambda is fine. So a lot depends on what you are doing in the lambda, how many you have, what your downstream service you are connecting to us.
paulgb 2y ago
One word of caution about naively porting a regular web app to lambda: since you’re charged for duration, if your app does something like make an API call, you’re paying for duration while waiting on that API call for each request. If that API breaks and hangs for 30s, and you are used to it being (say) a 300ms round trip, your costs have 100xed. So lambda pricing scales down to cheaper than a VPC, but it also scales up a lot faster ;)
- mlhpdx 2y ago
  Not untrue, but also not true. It isn’t a Lambda problem, It’s the problem of running the same old, same old code on lambda. Running clusters of VM/container/instances to “scale up” to transient demand but leaving largely idle (a very, very common outcome) then an event oriented rewrite is going to save a big hunk of money. But yeah, if you keep every instance busy all the time, it is definitely cheaper than lambda busy all the time.
  - paulgb 2y ago
    Sure, if you’re willing to rewrite for an event-based architecture you can avoid the problem, but the post is a framework designed for running on Lambda without that rewrite. One of the stated goals is to have the same container run on EC2 or Fargate.
    - mlhpdx 2y ago
      I understand the point. It’s one in a long line of such enabling tech — all leading to the same outcome.
    - zoover2020 2y ago
      A lot of internal web apps with low TPS can be easily ported to Lambda using these adapters. They should be the main use case IMO.
      - allannienhuis 2y ago
        A lot of internal web apps with low TPS can easily be run on a $5-30/month vps.
mlhpdx 2y ago
Help me understand how this could possibly result in a healthier SDLC. Great, it is possible to run the same container on all these services (and probably more given the usual quip about the 50 ways to run a container on AWS). And, I can see that being a tempting intellectual idea and even a practical “cheat” allowing understaffed orgs to keep running the same old code in new ways. But why? The less it changes the the more vulnerable it is, the less compliant it is, the more expensive it is to operate (including insurance), the less it improves and add features, the less people like it. Seems like a well-worn path to under performance as an organization.
- abadpoli 2y ago
  > The less it changes the the more vulnerable it is, the less compliant it is, the more expensive it is to operate (including insurance), the less it improves and add features, the less people like it. I don’t follow this. Complexity is what leads to vulnerabilities. Reducing complexity by reusing the same, known code is better for security and for compliance. As a security person, if a team came to me and said they were reusing the same container in both contexts rather than creating a new code base, I would say “hell yeah, less work for all of us”. There are other reasons why using the same container in both contexts might not be great (see other comments in this thread), but security and compliance aren’t at the top of my list at all (at least not for the reasons you listed).
  - mlhpdx 2y ago
    If the container has any dependencies, and they change, a lack of change quickly becomes an escalating liability. I’ve heard the “fixed is reliable” arguments for decades and have never seen anything that would justify the widespread failure to keep systems patched.
    - mlyle 2y ago
      He is not suggesting leaving the container unpatched, so this is unrelated criticism.
    - abadpoli 2y ago
ranger_danger 2y ago
How painful is iterating changes during development on this?
- inhumantsar 2y ago
  cycles can be pretty quick with lambda, definitely faster than launching a container into k8s or ecs. there's good tooling for local execution too
  - pluto_modadic 2y ago
    AWS codepipeline/codebuild still takes a really long time. think 1 minute to notice a commit, 2 minutes to download code, 3-8 minutes to get a codebuild runner, however long your build is, and 30 seconds to update the lambda. Especially when you don't have a good local AWS runner, vs using a HTTP service you can run and live reload on your laptop. deploying to prod in less than 10 seconds, or even /live reloading/ the AWS service would be awesome.
    - abadpoli 2y ago
      For fast iteration, our developers build, test and deploy locally using cdk deploy directly into their dev account, no need to wait around for the pipeline to do all those steps. Then when they’re ready, higher environments go through the pipeline. CDK has a new(ish) feature called hotswap that also bypasses CloudFormation and makes deploys faster.
      - chuckadams 2y ago
        The equivalent of hotswap in SAM would be `sam sync --code --watch`. Works in seconds, though I still prefer to test locally as much as possible.
    - deleted 2y ago
      [deleted]
  - nunez 2y ago
tills13 2y ago
Something I don't really get is that if you're going through the trouble of creating a container, why not just run it on fargate or ec2? Is it literally just the scaling the 0? And you're willing to give up some number of requests that hit cold-start for that?
- SahAssar 2y ago
  Scaling to 0 means me being able to deploy each branch/PR to a whole, fully separate environment without it costing too much. As soon as you do that everything from testing migrations (via database branching), running full end-to-end tests (including the full infra, db, backend, etc.), having proper previews for people to view when reviewing the changes (like for example the UX designer seeing the whole feature working including the backend/db changes before it going in) just falls into place. If I don't scale to 0 I'd prefer to work on dedicated hardware, anything in-between just doesn't give me enough benefit.
  - jiggawatts 2y ago
    You can get a similar effect with Azure App Service, which is basically a cloud hosted managed IIS web farm. A web app is “just” a folder and takes zero resources when idle.
    - deleted 2y ago
      [deleted]
    - ko_pivot 2y ago
      I wouldn’t recommend Azure App Service. Container Apps / AKS / Container Instances / Static Web Apps are all fine services for different use cases, but with App Service you are buying into a pretty strange infra approach compared to traditional FaaS / PaaS / IaaS distinctions. In the past 100% of the time App Service ended up being too weird to debug and another Azure service was a better compute fit.
      - jiggawatts 2y ago
sudhirj 2y ago
So the way we're using this is this: * We write our HTTP services and package them in containers. * We add the Lambda Web Adapter into the Dockerfile. * We push the image to ECR. * There's a hook lambda that creates a service on ECS/Fargate (the first-party Kubernetes equivalent on AWS) and a lambda. * Both are prepped to receive traffic from the ALB, but only one of them is activated. For services that make sense on lambda, they ALB routes traffic to the lambda, otherwise to the service. The other comments here have more detailed arguments over which service would do better where, but the decision making tree is a bit like this: * is this a service with very few invocations? Probably use lambda. * is there constant load on this service? Probably use the service. * if load is mixed or if there's are lot of idle time in the request handling flow, figure out the inflection point at which a service would be cheaper. And run that. While we wish there was a fully automated way to do this, this is working well for us.
- mfreeman451 2y ago
  [dead]
- wavemode 2y ago
  I guess the part I don't get is, if your traffic is that low, doesn't that mean you could run the service on a ~$4/month instance anyway? Even if we assume the lambda option is cheaper, it's only cheaper by pennies, basically. In exchange for taking on a bunch of added complexity.
  - bubblyworld 2y ago
    Not the OP, but things can be infrequent and bursty, or infrequent and expensive, or infrequent and memory-intensive. In those cases (and many more I'm sure) lambda can make sense.
  - sudhirj 2y ago
    Sort of, but two things add up - lots of little services (I don't like the microservices word), but we do make small services for the workloads that make sense to run and maintain independently. The other is burst, we run a business where customers send us a couple hundred thousand things to do just once a week or once a day. And of course we don't know exactly when the orders will come in. Much easier to let the lambdas run than have large running servers waiting. Autoscaling is possible, of course, but laggy autoscaling won't work, and aggressive autoscaling is basically lambda.
andrewstuart 2y ago
I've noticed any project that involves AWS requires a huge amount of "programming the machine" versus "writing the application". What I mean by "programming the machine" is doing technical tasks that relates to making the cloud work in the way you want. I've worked on projects in which tha vast majority of the work seems to be endless "programming the machine".
- acdha 2y ago
  That’s a choice, like using Kubernetes or being a micro service fundamentalist. A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data, and before that you had the J2EE types building elaborate multi server systems for what could have been simple apps. This repeats constantly because the underlying problem is a broken social environment where people aren’t aligned with what their users really need and that’s rewarded.
  - andrewstuart 2y ago
    I think it is because cloud computing demands far more programming the machine than deploying to compute instances does.
    - acdha 2y ago
      Does it, though, or are you just more familiar with traditional servers? There are a lot more things to manage in a traditional environment and things like tools are more primitive so you’ll have to invest a lot more time getting anywhere near equivalent reliability or security. That’s especially important if you have compliance requirements where you’ll be on the hook for things provided or greatly simplified by the cloud platform.
      - andrewstuart 2y ago
        >> There are a lot more things to manage in a traditional environment Can you be specific because that does not sound right to me. I know both pretty well and seems to me there is vastly more things that need to be tuned and managed and configured in cloud.
luke-stanley 2y ago
How does this compare to Fly.io which AFAIK wakes on requests in a very similar way with containers made into Firecracker VM's (unless using GPU, I think)? I suppose Fly typically doesn't need to scale per request so has a typical cheaper max cost? I guess you'd just set the concurrency but how well that scales, I don't know.
- luke-stanley 2y ago
  Vercel must do something a bit similar to Fly too, I assume.
deleted 2y ago
[deleted]
irjustin 2y ago
Question: this seems to take the port as the primary entry point of communication which means you need to be running a web server? This adds a lot of cold start overhead? Instead of directly invoking a handler.
- datashaman 2y ago
  It is a *Web* adapter.
- ben0x539 2y ago
  Why does a web server have significantly more cold start overhead than whatever is receiving requests and passing them to the handler in the classic lambda case?
nunez 2y ago
What's the difference between using this vs a Lambda function with an API Gateway app fronting it? I've been doing the latter for many years with Serverless and it has worked very well. Or does this enable you to run your entire web application as a Lambda function? If so, how does it work around the 30s execution window?
- cr125rider 2y ago
  This basically maps the incoming event to an ASGI request. You can create a typical web server and invoke it via function url and it will get mapped to your listening port
  - throw156754228 2y ago
    Are you talking about a lambda function url? Is there not a load balancer in between the lambda function invocation and the web server request?
throw156754228 2y ago
Does this mean Spring Boot for example has to be spun up and reinitialise itself for every request?
- renke1 2y ago
  Function instances should stay alive for a certain amount of time. But you will pay the cold start price one way or another, so naively using Spring Boot is probably not a good idea. Spring Boot Native might help here, but I haven't tried it yet.
  - throw156754228 2y ago
    But is the web server spun up and kept up somewhere magically? Another poster mentioned something about an ECS task. The reason I am asking is I genuinely don't understand, I thought lambda as an abstraction was all about something which spins up and down when finished.

134 points by cebert 2y ago

Thaxll 2y ago

Someone with experiance with Lambda, how does connection pooling works, are tcp connection re-used at all?

remram 2y ago
There's no direct connection to the Lambda container, it goes through a load balancer. The load balancer can keep connections open for reuse.
zyl1n 2y ago
As you might have guessed, connection pools don't work well with Lambda. For database connection pool, they came up with another chargeable service, RDS Proxy, to fix the problem created by Lambda :)
tiew9Vii 2y ago
A lambda lives for a relatively short period of time, ephemeral. It only handles a single request at a time but will live long enough to serve multiple requests. If another request comes in while one is in process a new lambda can spin up. As a lambda lives for a period of time you can create a pool for that individual lambda. That pool lives for the life of the lambda. Now for things like HTTP clients this is generally OK. If you have a Postgres pool etc, it's not ok. Postgress doesn't like lots of connections and you will have POOL_SIZE * NUMBER_OF_ACTIVE_LAMBDA connections. So unless you limit the number of Lambda's you have using reserved capacity Lambda's are simply not a good fit for running a PG pool locally inside the lambda for web stuff. If you have a batch job, that runs once a day, with a single lambda, inserts a bunch of records in to postgres or similar, then a small pool in the lambda is fine. So a lot depends on what you are doing in the lambda, how many you have, what your downstream service you are connecting to us.

paulgb 2y ago

One word of caution about naively porting a regular web app to lambda: since you’re charged for duration, if your app does something like make an API call, you’re paying for duration while waiting on that API call for each request. If that API breaks and hangs for 30s, and you are used to it being (say) a 300ms round trip, your costs have 100xed. So lambda pricing scales down to cheaper than a VPC, but it also scales up a lot faster ;)

mlhpdx 2y ago
Not untrue, but also not true. It isn’t a Lambda problem, It’s the problem of running the same old, same old code on lambda. Running clusters of VM/container/instances to “scale up” to transient demand but leaving largely idle (a very, very common outcome) then an event oriented rewrite is going to save a big hunk of money. But yeah, if you keep every instance busy all the time, it is definitely cheaper than lambda busy all the time.
- paulgb 2y ago
  Sure, if you’re willing to rewrite for an event-based architecture you can avoid the problem, but the post is a framework designed for running on Lambda without that rewrite. One of the stated goals is to have the same container run on EC2 or Fargate.
  - mlhpdx 2y ago
    I understand the point. It’s one in a long line of such enabling tech — all leading to the same outcome.
  - zoover2020 2y ago
    A lot of internal web apps with low TPS can be easily ported to Lambda using these adapters. They should be the main use case IMO.
    - allannienhuis 2y ago
      A lot of internal web apps with low TPS can easily be run on a $5-30/month vps.

mlhpdx 2y ago

Help me understand how this could possibly result in a healthier SDLC. Great, it is possible to run the same container on all these services (and probably more given the usual quip about the 50 ways to run a container on AWS). And, I can see that being a tempting intellectual idea and even a practical “cheat” allowing understaffed orgs to keep running the same old code in new ways. But why? The less it changes the the more vulnerable it is, the less compliant it is, the more expensive it is to operate (including insurance), the less it improves and add features, the less people like it. Seems like a well-worn path to under performance as an organization.

abadpoli 2y ago
> The less it changes the the more vulnerable it is, the less compliant it is, the more expensive it is to operate (including insurance), the less it improves and add features, the less people like it. I don’t follow this. Complexity is what leads to vulnerabilities. Reducing complexity by reusing the same, known code is better for security and for compliance. As a security person, if a team came to me and said they were reusing the same container in both contexts rather than creating a new code base, I would say “hell yeah, less work for all of us”. There are other reasons why using the same container in both contexts might not be great (see other comments in this thread), but security and compliance aren’t at the top of my list at all (at least not for the reasons you listed).
- mlhpdx 2y ago
  If the container has any dependencies, and they change, a lack of change quickly becomes an escalating liability. I’ve heard the “fixed is reliable” arguments for decades and have never seen anything that would justify the widespread failure to keep systems patched.
  - mlyle 2y ago
    He is not suggesting leaving the container unpatched, so this is unrelated criticism.
  - abadpoli 2y ago

ranger_danger 2y ago

How painful is iterating changes during development on this?

inhumantsar 2y ago
cycles can be pretty quick with lambda, definitely faster than launching a container into k8s or ecs. there's good tooling for local execution too
- pluto_modadic 2y ago
  AWS codepipeline/codebuild still takes a really long time. think 1 minute to notice a commit, 2 minutes to download code, 3-8 minutes to get a codebuild runner, however long your build is, and 30 seconds to update the lambda. Especially when you don't have a good local AWS runner, vs using a HTTP service you can run and live reload on your laptop. deploying to prod in less than 10 seconds, or even /live reloading/ the AWS service would be awesome.
  - abadpoli 2y ago
    For fast iteration, our developers build, test and deploy locally using cdk deploy directly into their dev account, no need to wait around for the pipeline to do all those steps. Then when they’re ready, higher environments go through the pipeline. CDK has a new(ish) feature called hotswap that also bypasses CloudFormation and makes deploys faster.
    - chuckadams 2y ago
      The equivalent of hotswap in SAM would be `sam sync --code --watch`. Works in seconds, though I still prefer to test locally as much as possible.
  - deleted 2y ago
    [deleted]
- nunez 2y ago

tills13 2y ago

Something I don't really get is that if you're going through the trouble of creating a container, why not just run it on fargate or ec2? Is it literally just the scaling the 0? And you're willing to give up some number of requests that hit cold-start for that?

SahAssar 2y ago
Scaling to 0 means me being able to deploy each branch/PR to a whole, fully separate environment without it costing too much. As soon as you do that everything from testing migrations (via database branching), running full end-to-end tests (including the full infra, db, backend, etc.), having proper previews for people to view when reviewing the changes (like for example the UX designer seeing the whole feature working including the backend/db changes before it going in) just falls into place. If I don't scale to 0 I'd prefer to work on dedicated hardware, anything in-between just doesn't give me enough benefit.
- jiggawatts 2y ago
  You can get a similar effect with Azure App Service, which is basically a cloud hosted managed IIS web farm. A web app is “just” a folder and takes zero resources when idle.
  - deleted 2y ago
    [deleted]
  - ko_pivot 2y ago
    I wouldn’t recommend Azure App Service. Container Apps / AKS / Container Instances / Static Web Apps are all fine services for different use cases, but with App Service you are buying into a pretty strange infra approach compared to traditional FaaS / PaaS / IaaS distinctions. In the past 100% of the time App Service ended up being too weird to debug and another Azure service was a better compute fit.
    - jiggawatts 2y ago

sudhirj 2y ago

So the way we're using this is this: * We write our HTTP services and package them in containers. * We add the Lambda Web Adapter into the Dockerfile. * We push the image to ECR. * There's a hook lambda that creates a service on ECS/Fargate (the first-party Kubernetes equivalent on AWS) and a lambda. * Both are prepped to receive traffic from the ALB, but only one of them is activated. For services that make sense on lambda, they ALB routes traffic to the lambda, otherwise to the service. The other comments here have more detailed arguments over which service would do better where, but the decision making tree is a bit like this: * is this a service with very few invocations? Probably use lambda. * is there constant load on this service? Probably use the service. * if load is mixed or if there's are lot of idle time in the request handling flow, figure out the inflection point at which a service would be cheaper. And run that. While we wish there was a fully automated way to do this, this is working well for us.

mfreeman451 2y ago
[dead]
wavemode 2y ago
I guess the part I don't get is, if your traffic is that low, doesn't that mean you could run the service on a ~$4/month instance anyway? Even if we assume the lambda option is cheaper, it's only cheaper by pennies, basically. In exchange for taking on a bunch of added complexity.
- bubblyworld 2y ago
  Not the OP, but things can be infrequent and bursty, or infrequent and expensive, or infrequent and memory-intensive. In those cases (and many more I'm sure) lambda can make sense.
- sudhirj 2y ago
  Sort of, but two things add up - lots of little services (I don't like the microservices word), but we do make small services for the workloads that make sense to run and maintain independently. The other is burst, we run a business where customers send us a couple hundred thousand things to do just once a week or once a day. And of course we don't know exactly when the orders will come in. Much easier to let the lambdas run than have large running servers waiting. Autoscaling is possible, of course, but laggy autoscaling won't work, and aggressive autoscaling is basically lambda.

andrewstuart 2y ago

I've noticed any project that involves AWS requires a huge amount of "programming the machine" versus "writing the application". What I mean by "programming the machine" is doing technical tasks that relates to making the cloud work in the way you want. I've worked on projects in which tha vast majority of the work seems to be endless "programming the machine".

acdha 2y ago
That’s a choice, like using Kubernetes or being a micro service fundamentalist. A decade ago, you had people building in-house Hadoop clusters for tiny amounts of data instead of doing their jobs analyzing that data, and before that you had the J2EE types building elaborate multi server systems for what could have been simple apps. This repeats constantly because the underlying problem is a broken social environment where people aren’t aligned with what their users really need and that’s rewarded.
- andrewstuart 2y ago
  I think it is because cloud computing demands far more programming the machine than deploying to compute instances does.
  - acdha 2y ago
    Does it, though, or are you just more familiar with traditional servers? There are a lot more things to manage in a traditional environment and things like tools are more primitive so you’ll have to invest a lot more time getting anywhere near equivalent reliability or security. That’s especially important if you have compliance requirements where you’ll be on the hook for things provided or greatly simplified by the cloud platform.
    - andrewstuart 2y ago
      >> There are a lot more things to manage in a traditional environment Can you be specific because that does not sound right to me. I know both pretty well and seems to me there is vastly more things that need to be tuned and managed and configured in cloud.

luke-stanley 2y ago

How does this compare to Fly.io which AFAIK wakes on requests in a very similar way with containers made into Firecracker VM's (unless using GPU, I think)? I suppose Fly typically doesn't need to scale per request so has a typical cheaper max cost? I guess you'd just set the concurrency but how well that scales, I don't know.

luke-stanley 2y ago
Vercel must do something a bit similar to Fly too, I assume.

deleted 2y ago

[deleted]

irjustin 2y ago

Question: this seems to take the port as the primary entry point of communication which means you need to be running a web server? This adds a lot of cold start overhead? Instead of directly invoking a handler.

datashaman 2y ago
It is a *Web* adapter.
ben0x539 2y ago
Why does a web server have significantly more cold start overhead than whatever is receiving requests and passing them to the handler in the classic lambda case?

nunez 2y ago

What's the difference between using this vs a Lambda function with an API Gateway app fronting it? I've been doing the latter for many years with Serverless and it has worked very well. Or does this enable you to run your entire web application as a Lambda function? If so, how does it work around the 30s execution window?

cr125rider 2y ago
This basically maps the incoming event to an ASGI request. You can create a typical web server and invoke it via function url and it will get mapped to your listening port
- throw156754228 2y ago
  Are you talking about a lambda function url? Is there not a load balancer in between the lambda function invocation and the web server request?

throw156754228 2y ago

Does this mean Spring Boot for example has to be spun up and reinitialise itself for every request?

renke1 2y ago
Function instances should stay alive for a certain amount of time. But you will pay the cold start price one way or another, so naively using Spring Boot is probably not a good idea. Spring Boot Native might help here, but I haven't tried it yet.
- throw156754228 2y ago
  But is the web server spun up and kept up somewhere magically? Another poster mentioned something about an ECS task. The reason I am asking is I genuinely don't understand, I thought lambda as an abstraction was all about something which spins up and down when finished.