7 ms·

From Express.js to AWS Lambda: Migrating Existing Node Apps to Serverless

90 points by slobodan_ 9y ago

fovc 9y ago
For people who run their apps like this, do you find the complexity is greater or lesser than running e.g. Express + nginx in richer containers? This looks very cool, but seems ripe for some ugly emergent behavior to appear.
- smt88 9y ago
  Node apps are incredibly easy to deploy to Elastic Beanstalk. For most users, the pros of Lambda are probably outweighed by the cons (even though the concept is fantastic).
- 0xcolton 9y ago
  Like all things, it depends. Your application architecture complexity is going to go way up, but your devops / sysadmin complexity goes to zero. I run a web hosting service, and while my application code is a bit more complicated now, I don't have to worry about monitoring ec2 instances, process managers, and that's worth it. That particular trade of is worth it. If you manage an application with a high level of application logic, such that keeping servers alive is relatively simple, maybe the calculus works out differently.
- nzoschke 9y ago
  After the initial “cost” of figuring out how to configure the services, I’m finding the complexity far less than containers. Many concerns are removed from my application code and build and deploy systems by Lambda and API gateway. You just throw some zip files in S3 and you have a web service. No more image registry, container orchestration, load balancers, AMIs, autoscaling, etc. You also get to remove things code around CORS, auth, polling etc. from your app. I have a bunch of writing on how FaaS makes things easier here: https://github.com/nzoschke/gofaas https://github.com/nzoschke/gofaas
diegorbaquero 9y ago
No async/await yet in lambda, it’s really sad. Serverless in AWS is stuck in 2015
- plumeria 9y ago
  You can use Webpack and Babel, in the meantime.
- outworlder 9y ago
  Node is not the only supported runtime. I'm happily deploying stuff in Golang, so no real runtimes to worry about.
- ihsw2 9y ago
  TypeScript accommodates async/await in <ES2017 environments.
  - joshribakoff 9y ago
    Typescript is a language that compiles to es5. Your comment makes it sound like typescript somehow makes an old runtime support new features. You can actually use any transpiler to produce older JavaScript, this is not unique to typescript, and you're shipping es5 code at the end of the day which has tradeoffs to consider
    - ihsw2 9y ago
      My comment was clear, anyone investigating the subject can confirm that polyfills are used to "make" an old runtime support new features (eg: using generators for feature-complete async/await on ES6, using a custom state-machine for feature-complete async/await on ES5/ES3).
- Can_Not 9y ago
  Most JS developers are already use to using webpack/Babel. On Babel's website is available some quick guides to starting a new project with many options/templates. Here is a project to enhance the lambda experience with webpack https://github.com/serverless-heaven/serverless-webpack/blob/master/README.md https://github.com/serverless-heaven/serverless-webpack/blob... In production you would probably want to minify and micro-optimize anyways, so why pick one of the standard build pipeline tools and get async/await also? Your last option is spotinst. If you use their serverless service, you get native access to node 8. They load balance across AWS EC2s, GCE, and Azure.
foob 9y ago
I'm kind of surprised that the article doesn't mention aws-serverless-express [1], a node module provided by Amazon which makes it fairly trivial to expose an existing Express app as a Lambda function via AWS API Gateway. The Lambda handler module basically just consists of the following code. const awsServerlessExpress = require('aws-serverless-express'); const app = require('./app'); const server = awsServerlessExpress.createServer(app); exports.handler = (event, context) => ( awsServerlessExpress.proxy(server, event, context) ); It's really convenient to be able to develop simple APIs using Express, and then to expose them via Lambda functions using aws-serverless-express. After you've done it once or twice, it's really easy to throw together single-purpose APIs and deploy them in a matter of minutes. For example, I was recently frustrated with the fact that CircleCI's API for accessing build artifacts is perpetually broken, and I wrote a little microservice to expose the same functionality [2]. It's definitely awesome to be able to deploy one-off projects like that without needing to worry about any sort of server maintenance. [1] - https://github.com/awslabs/aws-serverless-express https://github.com/awslabs/aws-serverless-express [2] - https://intoli.com/blog/circleci-artifacts/ https://intoli.com/blog/circleci-artifacts/
- Meegul 9y ago
  How's the latency when using this method? I'd imagine if you foresee your site being low usage, startup times would be quite bad. At least, on the order of seconds rather than milliseconds.
  - foob 9y ago
    It depends a little bit on whether there's a warm container or not, but I think that the overhead is much closer to ~100 milliseconds than to multiple seconds. The CircleCI artifacts API that I linked to takes less than one second to return an artifact, and there's a lot more going on there than a single request/response pair. The endpoint makes an external request to CircleCI's API, does some processing on the response, returns a 301 redirect, and then the redirect is followed by the client and another request is made to actually download the file from S3. You can test this directly by running time curl -L 'https://circleci.intoli.com/artifacts/intoli/exodus/coverage-report/total-coverage.json' which outputs something like the following (the timing will vary a bit obviously). { "coverage": "92.43%" } real 0m0.798s user 0m0.046s sys 0m0.010s
k__ 9y ago
Is pumping data really the right use of Lambda? I read it should be used for data transformatiom and computation and not transportation.
OnlyRepliesToBS 9y ago
Yes, you could take your Node app and fracture it to the quantum winds, ensuring Amazon a tidy profit off of you in perpetuity... BUT WHY WOULD YOU BE THAT STUPID?

90 points by slobodan_ 9y ago

fovc 9y ago

For people who run their apps like this, do you find the complexity is greater or lesser than running e.g. Express + nginx in richer containers? This looks very cool, but seems ripe for some ugly emergent behavior to appear.

smt88 9y ago
Node apps are incredibly easy to deploy to Elastic Beanstalk. For most users, the pros of Lambda are probably outweighed by the cons (even though the concept is fantastic).
0xcolton 9y ago
Like all things, it depends. Your application architecture complexity is going to go way up, but your devops / sysadmin complexity goes to zero. I run a web hosting service, and while my application code is a bit more complicated now, I don't have to worry about monitoring ec2 instances, process managers, and that's worth it. That particular trade of is worth it. If you manage an application with a high level of application logic, such that keeping servers alive is relatively simple, maybe the calculus works out differently.
nzoschke 9y ago
After the initial “cost” of figuring out how to configure the services, I’m finding the complexity far less than containers. Many concerns are removed from my application code and build and deploy systems by Lambda and API gateway. You just throw some zip files in S3 and you have a web service. No more image registry, container orchestration, load balancers, AMIs, autoscaling, etc. You also get to remove things code around CORS, auth, polling etc. from your app. I have a bunch of writing on how FaaS makes things easier here: https://github.com/nzoschke/gofaas https://github.com/nzoschke/gofaas

diegorbaquero 9y ago

No async/await yet in lambda, it’s really sad. Serverless in AWS is stuck in 2015

plumeria 9y ago
You can use Webpack and Babel, in the meantime.
outworlder 9y ago
Node is not the only supported runtime. I'm happily deploying stuff in Golang, so no real runtimes to worry about.
ihsw2 9y ago
TypeScript accommodates async/await in <ES2017 environments.
- joshribakoff 9y ago
  Typescript is a language that compiles to es5. Your comment makes it sound like typescript somehow makes an old runtime support new features. You can actually use any transpiler to produce older JavaScript, this is not unique to typescript, and you're shipping es5 code at the end of the day which has tradeoffs to consider
  - ihsw2 9y ago
    My comment was clear, anyone investigating the subject can confirm that polyfills are used to "make" an old runtime support new features (eg: using generators for feature-complete async/await on ES6, using a custom state-machine for feature-complete async/await on ES5/ES3).
Can_Not 9y ago
Most JS developers are already use to using webpack/Babel. On Babel's website is available some quick guides to starting a new project with many options/templates. Here is a project to enhance the lambda experience with webpack https://github.com/serverless-heaven/serverless-webpack/blob/master/README.md https://github.com/serverless-heaven/serverless-webpack/blob... In production you would probably want to minify and micro-optimize anyways, so why pick one of the standard build pipeline tools and get async/await also? Your last option is spotinst. If you use their serverless service, you get native access to node 8. They load balance across AWS EC2s, GCE, and Azure.

foob 9y ago

I'm kind of surprised that the article doesn't mention aws-serverless-express [1], a node module provided by Amazon which makes it fairly trivial to expose an existing Express app as a Lambda function via AWS API Gateway. The Lambda handler module basically just consists of the following code. const awsServerlessExpress = require('aws-serverless-express'); const app = require('./app'); const server = awsServerlessExpress.createServer(app); exports.handler = (event, context) => ( awsServerlessExpress.proxy(server, event, context) ); It's really convenient to be able to develop simple APIs using Express, and then to expose them via Lambda functions using aws-serverless-express. After you've done it once or twice, it's really easy to throw together single-purpose APIs and deploy them in a matter of minutes. For example, I was recently frustrated with the fact that CircleCI's API for accessing build artifacts is perpetually broken, and I wrote a little microservice to expose the same functionality [2]. It's definitely awesome to be able to deploy one-off projects like that without needing to worry about any sort of server maintenance. [1] - https://github.com/awslabs/aws-serverless-express https://github.com/awslabs/aws-serverless-express [2] - https://intoli.com/blog/circleci-artifacts/ https://intoli.com/blog/circleci-artifacts/

Meegul 9y ago
How's the latency when using this method? I'd imagine if you foresee your site being low usage, startup times would be quite bad. At least, on the order of seconds rather than milliseconds.
- foob 9y ago
  It depends a little bit on whether there's a warm container or not, but I think that the overhead is much closer to ~100 milliseconds than to multiple seconds. The CircleCI artifacts API that I linked to takes less than one second to return an artifact, and there's a lot more going on there than a single request/response pair. The endpoint makes an external request to CircleCI's API, does some processing on the response, returns a 301 redirect, and then the redirect is followed by the client and another request is made to actually download the file from S3. You can test this directly by running time curl -L 'https://circleci.intoli.com/artifacts/intoli/exodus/coverage-report/total-coverage.json' which outputs something like the following (the timing will vary a bit obviously). { "coverage": "92.43%" } real 0m0.798s user 0m0.046s sys 0m0.010s

k__ 9y ago

Is pumping data really the right use of Lambda? I read it should be used for data transformatiom and computation and not transportation.

OnlyRepliesToBS 9y ago

Yes, you could take your Node app and fracture it to the quantum winds, ensuring Amazon a tidy profit off of you in perpetuity... BUT WHY WOULD YOU BE THAT STUPID?