13 ms·
Going Serverless: Migrating an Express App to AWS API Gateway and AWS Lambda
- borplk 10y agoCan you achieve competitive latency for web pages using API Gateway and Lambda?
- ranman 10y agoI think acloud.guru is served through lambda. I think if you go the single page app route with S3 + cloudfront CDN + API gateway + lambda (so many words/things) you can deliver a really high quality product. You can also cache responses out via elasticache. It's not perfect but the development velocity and potential is really exciting.
- deleted 10y ago[deleted]
- ledgerdev 10y agoI was curious about the same thing and setup a jvm lambda to test. The setup was served through API gateway, connected to a postgres RDS, selected a couple records from a single database table and returned them as json, and deployed in us-west. From the west coast, I'm getting between 150ms and 300ms response times for most requests, but it does jump around quite a bit, with occasional requests taking 500ms-800ms to complete. A lot of the variability seems to be coming from execution time of the lambda itself, which would probably be because of the connection to RDS? I'm using HikariCP so the connection is pooled but maybe something like dynamo would be better. So latency is all over the place, but generally under half a second.
- taurath 10y agoConnection would only be pooled on a per-instance basis, which can be cached but sometimes are not for many parallel running processes (to my understanding).
- jomamaxx 10y agoListen folks - it's still a fantasy. Latency is still 5s on many requests = no deal. This has been going on for years. Would someone form AMZ please stand up? I suggest that the latency is 5s for the 'first call', then, subsequent calls are fast - and if you don't use your lambda for a minute or so, it goes back to having latency. I suggest maybe this has something to do with loading it into memory when it's used? There must be a way to keep the lambdas 'hot' or else it will never really get used for anything but background tasks.
- toomuchtodo 10y agoYou break the entire model of an oversubscribed multi-tenant container system if you want every lambda hot all the time. If that's what you need...you need a server.
- jomamaxx 10y ago"If that's what you need...you need a server." We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again. "You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.
- deleted 10y ago[deleted]
- toomuchtodo 10y ago> We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again. I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible. Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.
- 10y ago
- cycomachead 10y agoWherein "serverless" basically means more computers in more places just doing different tasks.
- iagooar 10y agoSome people prefer to call it Function as a Service, which in most cases is closer to what it actually is. "serverless" does not refer to the absolute lack of servers - it refers to the fact that your CODE (applications and software in general) should not be AWARE of being executed on a physical or virtual machine. The platform should be as abstract as it gets. I always imagine it as coding on a piece of paper: in a way, my code should work and make "sense", even though we all know that a piece of paper won't execute any instructions. And this is part of the magic of "serverless" software: it imagines a perfect, abstract machine where it can run, forgetting about the complex, unreliable, physical machine underneath. It's an almost philosophical approach to writing software and getting it needs some time and thought. And needless to say, it does not solve ALL your problems, but it MIGHT solve SOME of your problems.
- stephenr 10y ago> it refers to the fact that your CODE (applications and software in general) should not be AWARE of being executed on a physical or virtual machine. With the exception of para-virtualised device drivers, what code is ever aware of being executed on physical or virtual machines?
- windowsworkstoo 10y agoC'mon now, you know what was meant...
- stephenr 10y agoNo, I honestly don't, using that definition. AWS Lambda and similar services have literally nothing to do with hardware vs software. They're about running a function in response to an event on someone else's stack, almost like the `map` part of a `map-reduce`, if the input was considered to be distinct events/objects.
- hirenj 10y agoI have just recently got through a migration from express to a lambda based stack. In the end, performance is pretty good (once the lambdas are warm), but there were a few tricky parts, but nothing impossible. Serving backend requests is easy with Lambda, but the code for managing the backend can get quite large, quickly. If you are loading up data and it's going to take more than 5 minutes to perform a task, you need to build a queueing system from AWS primitives.
- markonen 10y agoIf anyone's interested in doing the math about the costs involved, I have two quick notes: first, the API Gateway, at $3.50 per million requests, is typically more expensive than the Lambda functions themselves (and comes with no permanent free tier, unlike Lambda); second, the 128MB minimum memory for a Lambda function can be on the tight side for an Express app, so you might want to base your calculations on more memory.
- gjolund 10y agoI love lambda's, but api gateway was far from mature the last time I used it (6 months ago). Unless they have made serious updates to the deployment workflow, feature set, and documentation I would not recommend building your products on it.
- simonpantzare 10y agoThey released proxy resources to API Gateway. Lets you specify a route like /foo/{path+} and all requests are forwarded as-is.
- davidjnelson 10y agoGreat for simplifying universal web apps.
- jotto 10y agoThe "0 to 1" workflow right out of AWS could not be easier, they have an in-browser code editor with a hello world already written for you. But that workflow doesn't make sense for real dev teams. So there's https://github.com/serverless/serverless https://github.com/serverless/serverless . It's driven by a yaml file and is intuitive. The only thing I'm bothered by is lack of environment variables (have to hard code them) and terrible API Gateway latency - see https://news.ycombinator.com/item?id=12681926 https://news.ycombinator.com/item?id=12681926 further down the page
- pmontra 10y agoFor js projects you can work around that problem with const config = require('./config.json'); Then you do config.secrets.apiKey etc You have to gitignore config.json and include it in serverless.yml Other languages might have to read and parse that JSON into one of their data structures.
- joeyspn 10y agoHave you seen https://github.com/silvermine/serverless-plugin-write-env-vars https://github.com/silvermine/serverless-plugin-write-env-va...?
- erikcw 10y agoI've recently deployed a small Flask based micro-service for an internal tool leveraging Zappa[0] to deploy it to Lambda/API Gateway. Overall it's been a really smooth experience. I'm looking forward to using it again. [0] https://github.com/Miserlou/Zappa https://github.com/Miserlou/Zappa
- ma2rten 10y agoIs there a big overhead in terms of latency? If I understand correctly you essentially have to start a server for each request.
- Mizza 10y agoZappa author here - thank you for your support!
- ranman 10y agoI love zappa, there's also chalice: https://github.com/awslabs/chalice https://github.com/awslabs/chalice
- sdrothrock 10y agoHave you used both? How would you compare/contrast them?
- doublerebel 10y agoSo, Apigee had this exact service. They did something pretty cool, it seems they implemented nearly all (!) of the Nodejs APIs in a JVM variant, then designed them to run on demand [0]. I actually ported an Express app to Apigee's service, to run webhooks and ETL. It worked flawlessly. The one downside was that they kept their own internal registry of compatible packages and there was no way they could keep up with the Nodejs ecosystem even with few holes in their API. I still have a ton of hope that Google does something smart with this, I actually wanted to pay Apigee for their service, but their pricing started at an Enterprise tier and none of my projects made it that far.... yet. Something to think about. [0]: http://docs.apigee.com/api-services/content/overview-nodejs-apigee-edge http://docs.apigee.com/api-services/content/overview-nodejs-...
- swang 10y agoi hope they upgrade to node 6 soon. i can't use fancy destructuring and some other es2015 functionality!
- davidjnelson 10y agoYou can use Babel for server side code too which would get you what you want.
- neximo64 10y agoI hope they solve the latency issues for a basic function that returns text (a basic callback(null, 'hello world') with the api gateway that is running 'hot' with the container model and yet still has so much latency.
- deleted 10y ago[deleted]
- deleted 10y ago[deleted]
- Everhusk 10y agoI'm always surprised at the lack of discussion around database connection reuse with AWS Lambda. It's a pretty big deal that every single API call requires a new database connection. The only solutions I've seen so far are to run a separate app to interface with the database, or moving the connection outside of the handler (which still has issues). Am I missing something or is everyone just really happy to use DynamoDB?
- Mizza 10y agoIf you design your correctly (at least in Python), you can cache and reuse your open database connection.
- ranman 10y agoHow exactly have you been doing that? It would be a cool blog post if you had the time. I've been thinking about really high volume applications that connect to haproxy instances that do DB load balancing but even that has it's own set of initial latency issues. I haven't fully figured out that architecture. In python in particular I'd really love to know how to setup the connection reuse.
- Mizza 10y agoCheck out the handler source code in Zappa for an example of a pattern like that. Similarly, if you use Zappa to deploy your application, if you create your database connection when the application loads, it'll just work. Stop by the Zappa slack if you want to explore this in more detail! https://slack.zappa.io https://slack.zappa.io It could always use more investigation, but there are quite a few Zappa users at extremely high loads now without any issues.
- avyfain 10y agoThis project looks super interesting. Thanks for sharing it! Now I want to come up with a use case to test it out over the next few days, maybe something for the Echo? I joined your Slack, we'll see where this takes us :)
- partycoder 10y agoI hate the term serverless. It involves a server. AWS Lambda is a service bound to a port therefore a server. It runs user defined functions, true. But it is a server.
- PetahNZ 10y agoIsn't it more of a cluster rather than a server
- partycoder 10y agoA cluster of? servers. Again, server. server, server, server. Repeat with me: server.
- HammadB 10y agoIt's just semantics, point is, you don't have to worry about spinning up servers, provisioning them, or maintaining them; just write code.
- partycoder 10y agoWrite code to run on? a server.
- blowski 10y agoCall it whatever you want, but I like to use the language of those who hold the purse-strings. If they call it "serverless", then I call it "serverless". If delivering a lecture about why it's not really serverless improves your product, then go ahead and deliver that lecture, but I doubt it will.
- partycoder 10y agoInvolves a server listening on a port, running on infrastructure that you indirectly pay. Not in the air, on a server. It is a very misleading term. Especially for non-tech savvy. Sounds like a scam. It's like calling being passenger on a car "wheel-less transport", because I as passenger don't have wheels.
- siscia 10y agoA little of a shameless plug, but I am the author of effe GitHub.com/siscia/effe A microservice framework. If you are worried about latency or you want lambda on your own servers it may be worth to take a look. Feedback welcome
- halmor 10y agoI am currently using AWS API Gateway + Lambda for the serverless backend of a social traveling startup. It is absolutely awesome: we are few people in IT and it is amazingly simple to manage the whole stuff. Moreover imho they are mature services at this time, they are flexible enough with enough configuration properties to fulfill a nearly full control of the development experience.
- lucaspiller 10y agoHow much does it end up costing you? Any ideas how much a 'traditional' setup would be in comparison?
- pksunkara 10y agoWe reduced our service server costs to 1/6th by moving from Heroku to Lambda. And that's not even considering the main benefit of a big computational request not blocking other requests (which is why we moved to it in the first place).
- twagner 10y ago@hobofan, take a look at our recent API Gateway features and see if the greedy paths and pass-through settings provide what you're looking for (they're also supported by CloudFormation). We tried to simplify the "configure every route" problem, but always looking for additional suggestions to make API config easier. @jomamaxx (& others): Reducing latency (and latency variability) is a critical goal for our team. We've improved p99 variability in API Gateway-related latency over the last few months, and will be addressing some of the Lambda-related latency that occurs when we refresh containers in the coming months. Additional latency optimizations coming at all levels of the stack, including networking. A clarification on the discussions about "managed hosting": Lambda is not classic web hosting; in fact, we block Lambda functions from calling socket.listen. In the Lambda model (whatever you prefer to call the broader category), the cloud service sees every request in order to perform scaling and load balancing on the function's behalf. Happy to chat offline with anyone who has additional questions or feedback: DM me on twitter @timallenwagner
- scrollaway 10y agoTim, when are we getting Python 3 support on Lambda? Being stuck on 2.7 has been the greatest source of issues and the #1 problem we have with Lambda. If Google/Microsoft came out today with Python 3 support in their Lambda competitor, we'd move in an instant.
- rurounijones 10y agoPiggybacking above comment with "Wheres da Ruby?" :D Using shims or jruby (with slow startup wasting milliseconds) hurts.
- noobiemcfoob 10y agoI'm in the same boat. We're developing a new product with no real reason to stay on Python 2, besides Lambda. Python 3 support would make my life so much easier.
- twagner 10y ago
- gtrubetskoy 10y agoOne significant problem with the API Gateway / Lambda integration is that the protocol they use to communicate is JSON. JSON is a text-only encoding, i.e. anything that isn't valid UTF8 isn't valid JSON, and so if you need to pass something binary to/from the Lambda (e.g. an image), you need to base64 encode it, which I'm actually not even sure is at all possible with the API/GW templates. There are other issues, but this alone is a huge headache if you ever have to work around it.
- falcolas 10y agoServerless via Lambda has been, frankly, a disappointment so far. The benefit is that I'm supposed to not have to manage servers anymore, yet I find myself, well, managing servers. I have to build and assign IAM roles, subnets, set up application configuration files in fixed external services (since you can't do things like set environment variables), configure the endpoints in nginx^h^h^h^h^h API Gateway, suffer through cold starts, fight against arcane packaging issues with site directories and .pth files (though this is probably just be a Python thing), fight concurrency throttling, external timeouts, manage out-of-band database connections, logging which requires even more IAM permissions... This doesn't even consider how many times I've had to tear down and re-create an API Gateway setup because it got "stuck" and would stop working. Finally, WTF is up with not being able to test API Gateway -> S3 integrations if the S3 bucket is in the same region as the gateway instance? It's a two plus year old bug by now, and a real pain (especially when coupled with other AWS services which require the bucket to be in the same region as the rest of a service - such as CodePipeline). Perhaps it's just because I'm familiar with setting up and provisioning servers, but for basic web services and periodic tasks, Lambda is much harder to work with most of the time. Its biggest benefit so far has been the low cost.
- noobiemcfoob 10y ago> Perhaps it's just because I'm familiar with setting up and provisioning servers I'd be inclined to say that may be the case. Our codebase leans heavily on Lambda, primarily as a wide stage in our data pipeline. And I love it. Using a configuration file in S3 - or any other means of dropping a text file on AWS - is barely more complicated than open("file.txt"). The true frustration is when you need to work with files beyond your memory limits, but even that can be mitigated by divide and conquer approaches. That said, the API Gateway is annoying in that it makes complex things impossible and simple things more configuration than they need. Yet, I'd still take it over actually standing up a server.
- falcolas 10y ago> is barely more complicated than open("file.txt") Well, that depends. First, you have to set up the IAM roles to access the bucket which owns file.txt. Then you have to ensure that "file.txt" is properly shared between accounts if you separate production from staging from development. Then there's three lines of code to set up a client, pull the file, and read the contents. If it's an encrypted file, you also get to do a bit more setup work to ensure that you're using the v4 signature in the library, since it defaults to v3 signatures. There's also the additional error handling and points of failure to account for. The requests are far from instantaneous as well, so you need to set up an out-of-band cache for the contents of the file if you plan to use it with every call of the lambda function. Is it obtusely hard? Of course not. Is it as simple as `open("file.txt")` (or more appropriately `os.environ["FOO"]`)? Not by a long shot. I have a feeling that our differences in opinion have more to do with the environment in which we run our lambda functions than what the lambda functions are doing. Once you move beyond a single region on a single account, things get a lot harder.
- JohnnyConatus 10y agoHow does this work, if at all, for local development? Is there a pseudo-Lambda server that runs locally? If not, this introduces yet another environment variable or it requires changes to be pushed constantly to Lamba for test.
- breandr 10y agoCheck out our example on github https://github.com/awslabs/aws-serverless-express/tree/master/example https://github.com/awslabs/aws-serverless-express/tree/maste.... The lambda wrapper is very thin (4 LOC). So you have two options: run/test your express app locally as you always would, or use the provided `npm run local` command which simulates the API Gateway+Lambda part (you can modify the `api-gateway-event.json` to change the "request" from API Gateway). This is primarily an example and starting point, and there is much more you could do to improve this process.
- vamur 10y agoWhat is the benefit of AWS Lambda vs getting a cheap VPS or a dedicated server with much better specs and bandwidth, setting it up and adding auto-update? And how does one prevent going bankrupt if a Lambda app is DDOSed?
- falcolas 10y agoThe cost is stupidly low. You would really have to work hard to go bankrupt. That said, there is a concurrency limit, which would help prevent your server time going to exponential limits.
- vamur 10y agoArticle doesn't mention any numbers. OVH VPS is $3.50 for a 2GB RAM and 100mbs (about 14TB a month assuming 50% utilization). That is about 14mln pages with 1MB size vs 1mln requests for API Gateway at $3.50, not including Lambda costs or bandwidth costs.
- falcolas 10y agoI don't have exact numbers, but we have dozens of lambda functions written in Python which are parsing and acting on Kinesis log streams (at volumes that cause problems for the Splunk ingest), and regularly hit the 100 process concurrency limit... but our bill is well under $10 a year (yes, year, not month. Hence my use of the adjectives "ridiculously" and "stupidly"). Luck? Hidden discounts? Don't know. Just know that our bill is ridiculously low for everything we're doing.
- hehheh 10y agoAuto updates can break shit; you need to monitor and understand the output of the auto update script. That understanding doesn't come cheap.