6 ms·
Serverless Best Practices
- tomcam 8y agoI still don’t understand how you do a PostgreSQL insert on a serverless system—please help! Also, this statement is patently silly: > The biggest point to make here is that serverless architecture may well require you to rethink your data layer. That’s not the fault of serverless. Well, it is the fault of serverless. It’s a shortcoming. Own it. The trade off may we be worthwhile, I just can’t tell yet. I’m trying to wrap my mind around serverless as regards database-backed apps, but obviously it would be preferable to be able to use the RDBMS of my choice.
- matchagaucho 8y agoMoving away from RDBMS just wasn't an option for us. But AWS serverless provided a couple benefits: 1. Aurora Serverless RDBMS (In pilot) 2. Lambda function property caching (for which we cache connections)
- staticassertion 8y agoHow does Aurora Serverless help with the issue of having to reconnect for every lambda? (I am unfamiliar with the Aurora Serverless offering) Also, can you elaborate/ point me to info on 2?
- ersiees 8y agoYou don’t need a reconnect for every invocation actually, only per instance. I know you don’t want to think about instances with lambda, but actually they just run on some VM and you can just open one connection per VM.
- lostcolony 8y agoReally quickly then - Background info: A lambda that spins up stays live for a while, and can continue to handle requests. The function handler is called again, so anything scoped to it, will disappear with the function, but anything scoped outside of the function will persist. Ergo, move your connection creation outside of your handler. Realistically you may need some abstraction to properly close and reopen the connection in the event of failure (so you can't have a 'bad' lambda, where something has happened to the connection and you can't fix it without deploying code again, to trigger a lambda refresh), but for the most part it just works.
- staticassertion 8y agoInteresting. I'd considered this, but was worried about failure cases. Seems tricky.
- philliphaydon 8y agoKeeping connections open on a lambda isn’t a good thing. If you have a lot of Lambdas and a lot of load you can end up hitting max connection limits. Better to open late and close early and take a hit rather than risk taking your db offline.
- lostcolony 8y agoOh, agreed; it doesn't bottleneck on the connection but instead keeps creating them which can take performance down when you spike. Hence why the original article speaks of using something other than a RDBMS, such as Dynamo, since there really is no way of addressing that issue (since you can't set a concurrency limit on one kind of function; else you could have one function that has high concurrency invoke one that is far lower, that exists just to keep connections pooled). If you absolutely must hit an RDBMS, be quick about it to free up the connection, and definitely consider if the latency matters that much; a few dozen ms more to initialize and close the connection is going to be a LOT cleaner. But that said, how badly it penalizes you for scaling connections up depends on the DB.
- tachang 8y agoI agree with this. Having a traditionally relational store can be a huge help: lots of people know it well, the disadvantages are well known, and everything is insanely mature. I dislike how the article tries to say that isn’t serveless’s fault when it is just a trade off.
- actuator 8y agoYeah, RDBMSs do have some big advantages; one of them being the plethora of query patterns possible. If there are other parts of your application which are consuming the same data but in different patterns, it might not be necessary that systems like DynamoDB work in those cases. You might need to make several secondary indexes in those cases and all of them require provisioning of their own. The other alternative being setting up some ETL layer which transforms the data and sends to appropriate sinks, from where they can be consumed; but for small services that adds more moving parts and cost to the system.
- deleted 8y ago[deleted]
- deeg 8y agoAgreed. Perhaps the first thing a "best practices" post should start off with is what kinds of problems serverless is best suited for.
- travbrack 8y agoSo why is rdbms and serverless seemingly mutually exclusive? Are rdbms queries really that slow compared to something like dynamodb?
- actuator 8y agoActually RDBMS queries are quite fast. But those systems weren't designed for use patterns which serverless applications typically have. Like MySQL will limit you on connection count and each new connection puts extra overhead on the server. So if every function negotiates its own connection you might eat away DB resources just on negotiating connections even though the DB queries were fast.
- k__ 8y agoAs far as I know, most RDBMS are connection based, which brings an overhead. When you got your connection they're quite fast.
- whopa 8y agoI wonder why all the predominant RDBMSes are heavyweight connection based, and why this is presumably so hard to change. There's nothing in SQL that requires this to be the case, at least conceptually.
- k__ 8y agoSaves overhead. Crappy application-servers pump queries like nobodies business, so you need to shave every byte of them you can.
- pvg 8y agoPart of it is legacy but also, interactions with an RDBMS are often stateful so it's useful to have a session (in which such things as transactions can live, etc). I'd guess there isn't much reason to change this because there are standard and effective workarounds that just haven't yet made their way to things like lambda.
- k__ 8y ago> It’s a shortcoming. Own it. Well, I don't know much about Oracle and the other players, but AWS at least tries to own it with Aurora Serverless.
- ActVen 8y agoWe use lambda functions(node with pg) to process incoming data. We just connect them to Aurora(Postgres) and make the inserts. We have never had any problem with the process whether we are pulling from a queue or other data source.
- languagehacker 8y agoBig ups for calling out how bad an idea connecting to the database is from a serverless context. I do a lot with serverless on AWS Lambda. Since part of that involves an ETL to write to a database, I also run a server that exposes a service for writing data via RPC. The server provides a connection pool to the database and appropriately encapsulates all the complex functionality we have for incoming data behind a well-defined interface. If you really insist on calling your entire stack serverless, do what we do and run these servers as ECS tasks via Fargate. That is plausibly serverless, albeit long-running. You get all the perks of a serverless environment with all the perks of something like beanstalk (without having to patch a server!). The drawback in this scenario is that running ECS tasks via Fargate don't provide as much flexibility in cost.
- Agebor 8y agoConnecting a DB might be a bad idea only currently, until cloud providers implement some sort of RPC SQL layer. We are successfully running Lambda functions connecting to AWS Aurora MySQL using IAM authentication. It has some quirks but after figuring it out, works well. Just run a connection pool with only 1 connection and expect some additional latency on cold start, though it's negligible compared to loading libraries, runtime, etc.
- zbentley 8y ago> implement some sort of RPC SQL layer > Just run a connection pool You just described the high-level architecture of what I believe is the majority of web application servers and the applications they run: an RPC layer over database accesses that run on persistent connections.
- tnolet 8y agoI use a specific Lambda function that acts like a sink. It receives messages via SNS from a myriad of other services and its only job is to push these messages to the main applications API. There the actual inserts and updates are done. The main API lives on Heroku.
- deleted 8y ago[deleted]
- Agebor 8y agoI would rather disagree with these practices. Some of them are subjective and some will likely be outdated soon. > Each function should do only one thing. The problem with one/a few functions running your entire app, is that when you scale you end up scaling your entire application. Sure, but is that a problem in general? Same thing can be said about a monolith vs microservices, and there is always a trade-off. A function with a larger code-base makes the code easier to navigate; may be easier to deploy. First reason about / compare cold-start times before making the decision to break the function. > Functions don’t call other functions Is the cost of calling the other function negligible (e.g. the functions are called rarely)? If yes, feel free to do it. > Use as few libraries in your functions as possible (preferably zero) Security risk is another discussion, not related to serverless. Worry instead about the cold-start time. If it's short enough for your use case - do use libraries. Should be no problem for Go, Node is usually OK. Java - native compilation is slowly approaching. > Avoid using connection based services e.g. RDBMS If adding another library is not a problem in your case, then why not? It's not too difficult to create a pool with a single connection. As for security, there are some new options like AWS Aurora IAM authentication. No VPC needed.
- tylerhou 8y ago> Functions don’t call other functions I think this point may might also be about reducing complexity, not just cost. Pushing to a queue instead of directly calling another function decouples the two functions and might make it easier to reason about how logic flows.
- Agebor 8y agoTrue, might make it easier in some cases. This is a general architecture decision, unrelated to serverless.
- k__ 8y ago>Is the cost of calling the other function negligible (e.g. the functions are called rarely)? If yes, feel free to do it. The question is more one of errors, I think. You pump your results in a queue before you let another function do the next step, if one function fails it's not the whole process that is lost. Step Functions, for example, lets you chain functions and does retries for you if one function failed.
- actuator 8y agoHe mentions DynamoDB for the data layer instead of regular RDBMS. I have tried to use DynamoDB several times but have been disappointed with it. I am not saying traditional RDBMSs solve the problem that DynamoDB has, but even DynamoDB is a poor fit for systems with rapidly changing load patterns. Some things that have been big pain points while using DynamoDB: - It has autoscaling built in now but even that leads a lot to be desired, autoscaling is slow and your lambda functions will error out for significant amount of time(I have seen it happen upto 5 minutes) for 5X spikes until it scales up the IOPS. This fills up the queues quite fast and can lead to patterns where your spike increases because of failure retries. You can control this somewhat by limiting lambda invocations though. - Limited number of scale downs allowed. - Scale down doesn't completely reduce the throughput limits. I have seen situations where the write load went to 0 on certain times but scale down reduced it to something like 400 IOPS which can be a huge cost drain. I am still looking for a DB which has done this well. Essentially what I want is compute layer separated from the storage layer in the database well enough so that both can be scaled independently and the compute layer can react to quick changes in load patterns. Does anyone here have any recommendations?
- Agebor 8y agoNot sure what do you mean about scaling storage layer separately from the compute layer. It's how it's usually done currently, with the disk being the storage layer. As long as there are machines/containers to be started, there will always be some latency, though we can expect it to improve in the next years. AWS now has the Aurora Serverless, though I don't have any real-life experience with it's latency yet (VPC requirement is a downer for now). https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.how-it-works.html https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...
- actuator 8y ago> Not sure what do you mean about scaling storage layer separately from the compute layer. It's how it's usually done currently, with the disk being the storage layer. Yeah, I meant that only. DynamoDB and Aurora Serverless are examples where they are decoupled and can be scaled independently. So, I wanted to know about more products like them but ones that scale with load really well. > AWS now has the Aurora Serverless, though I don't have any real-life experience with it's latency yet (VPC requirement is a downer for now). I have tried Aurora Serverless and I like it but it also plagued by the provisioning problems for now. The lack of options to customize database parameters was kind of a downer. Also for some weird reason they only support MySQL in t type instances. > (VPC requirement is a downer for now) BTW, you have an use case where you need to use DB from outside VPC and you can't do peering?
- starkingclojure 8y agoAre there any major success stories of big projects going serverless? I like the idea, but personally it mostly appeals to me as a nice way to reduce maintenance overhead for side-projects rather than something I'd use for a serious project.
- lostcolony 8y agoiRobot (Roomba)'s are pretty heavily serverless. https://thenewstack.io/irobot-confronts-challenges-running-serverless-scale/ https://thenewstack.io/irobot-confronts-challenges-running-s...
- GuB-42 8y ago> If you’re not aiming to scale that far, then you can probably get away without following these best practices anyway. If you don't intend to scale beyond what a server can offer (which is a lot!), you probably shouldn't be "serverless" in the first place.
- k__ 8y agoNot having to set up and maintain your own web/app/db-servers is quiet a nice thing to have, especially for small companies that don't have much money.
- deleted 8y ago[deleted]
- scarface74 8y agoTrue and that’s completely orthogonal to Serverless. Use RDS for DB servers and just use Elastic Beanstalk. It takes most of the complexity out of deploying load balanced web servers. Serverless for app servers is sometimes fine if you don’t need real time responsiveness like working with queues.
- k__ 8y agoYes, I probably wouldn't use FaaS stuff like Lambda for an API, but AppSync.
- manigandham 8y agoThese are all outdated already. Serverless is a meaningless term. The real name is platform-as-a-service, which is decades old. Running individual functions was just taking that to an extreme but any complex app will have multiple functions working together so it's right back to the same thing. Many "serverless" environments are in fact converting to running an arbitrary docker container for as long as it needs to run, effectively becoming the next-generation of PaaS where you container can be whatever size you need, while still not worrying about infrastructure and individual servers.
- a_silly_name 8y agoI'd appreciate it if others could chime in on this one. I'm mystified by the word 'serverless'. Why are we using this word? There are clearly, _quite clearly_ servers involved. The wacky bugs involved in these 'server-less' services... are going to come down to what's happening on the servers involved. EDIT: my question seems to have offended. Unfortunate, and unintentional!
- grzm 8y agoServerless refers to you administering the application and its resources but not the server directly. Yes, at some level servers are involved, but generally the organization deploying applications is not responsible for administering the hardware, the OS, other resources, often including the scaling of the server resources: they're only concerned with the administration of the application. One can argue that at some level the organization deploying the application is often still concerned with the underlying server and where that line is drawn, but many find it meaningful to make such a distinction. https://en.wikipedia.org/wiki/Serverless_computing https://en.wikipedia.org/wiki/Serverless_computing
- a_silly_name 8y agoDoes make sense, but your description (and others) seems to match most vendors API out there today. It's not clear to me what 'serverless' means vs 'hosted / managed infrastructure', or even 'external API'. It doesn't seem like a technical term at all. Another commenter mentioned that it applies only to the price list, and that makes sense to me.
- AYBABTME 8y agoThis one will get me into the most trouble. A lot of web application people will jump on the “but RDBMS are what we know” bandwagon. It’s not about RDBMS. It’s about the connections. Serverless. works best with services rather than connections. Guess what a service call will use to do its RPC...? a connection. This advice makes no sense. Maybe you don't care about the response to your service? That's different from saying "don't use connections". (I doubt most serverless RPC calls are over UDP like protocol)
- staticassertion 8y ago> Functions have cold starts (when a function is started for the first time) and warm starts (it’s been started, and is ready to be executed from the warm pool). Cold starts are impacted by a number of things, but the size of the zip file (or however the code is uploaded) is a part of it. Also, the number of libraries that need to be instantiated. I'm curious what size packages people are loading in lambdas currently. My largest lambda is a 4.7MB zip. If anyone has some anecdotal info on runtime + package size I'm super curious to hear. > So if you have to use an RDBMS, but put a service that handles connection pooling in the middle, maybe an auto scaling container of some description simply to handle that would be great. Not sure I understand this. If I put another service in between me and the database don't I just end up having to open connections to that service? I have an Aurora instance my lambda talks to, and it creates a new connection every time, and that's a bummer so I'm curious about how I could optimize that patch.
- k__ 8y agoThe service could be stateless.
- staticassertion 8y agoYeah, I wasn't thinking about what goes into a connection to a DB vs a stateless service. Thanks.
- glckr 8y agoConceptually I would say I'm a fan of these sorts of ideas (serverless, and queues in particular). Forcing you to look at the system as a chain of processes operating on data can really bring architectural problems into line. However, 99% of the work that I've done involves users hitting buttons and us responding to them synchronously. In these scenarios, I simply can't figure out how queues (and chains of serverless functions as advocated by this blog) are supposed to work (if they are at all). There seem to be many ways to solve this when the queues are all flowing freely, but as soon as there's any sort of pressure on the system these things all look to fall down. Looking at the amazon booking flow as an example -- it appears that they always show a "your order has been placed" page with a big green banner synchronously at the end of the cart flow. Some time later the user may then receive an email saying their payment method was declined. This certainly works, but a) it's horrible UX and b) it only works at the final stage of the process. I see queues (and serverless) advocated as good architectural decisions, but every time they come up in a lecture/blog they're given in toy or data-sciency sort of examples. Is it possible to use these patterns in a sensible way where users are actually involved? (the blog mentions CQRS, but that seems... not a perfect solution)
- krn 8y agoAs long as people talk about a development practice only in terms of a set of services from a proprietary cloud platform, you know, that it's not a mature thing.
- ryanmarsh 8y agoI don’t follow the logic in this argument. The same patterns work in other cloud platforms.
- krn 8y agoServerless applications are attached to a single cloud platform they were built on top of. You can't switch between the platforms without changing the code. At its current stage, serverless is more like a hack.
- lostcolony 8y agoSo is every managed service then; in AWS almost everything uses IAM and that's not portable. I can't use ECS; that's not portable. I can use EC2s, those are just VMs...except how the heck do I build those? I can't do it by hand, that's not portable, I can't use Cloudformations, that's not portable...I can't even use Terraform, as despite their marketing speak I still have to change out my configs, because they're still cloud dependent. Really, it's all a sliding scale. Buying into any cloud, even just at the VM level, means you've accepted your automation tools are going to be platform specific, and require changing recipes/configs/etc to go elsewhere. If you want to leverage anything beyond that, such as object storage (S3 in AWS, which is super common), then your code has to become aware of IAM and S3 endpoints. Your code now has to change if you want to deploy it elsewhere. I would contend your definition needs to be changed if the logical application of it basically makes everything more complex than 'someone else's server' to be "a hack"
- krn 8y agoA public cloud only requires three things: compute instances, block storage, and object storage. Everything else is just an additional service built on top of these. If you use a cloud platform as an infrastructure provider, not as a software provider, there is no vendor lock-in. There are enough mature open-source tools even to build your own cloud on bare metal[1]. [1] https://maas.io/ https://maas.io/
- robertonovelo 8y agoIMHO RDBMS and libraries are OK, just make sure you know the tradeoffs and have a strategy to withstand any potential scale issue or trouble. Do not reinvent the wheel, build for scale when it's needed, know both your function's and system scope/scale, etc.
- jmull 8y agoThis probably should be retitled “why you shouldn’t use serverless for most things”. You would only need to slightly rephrase the sentences introducing each section. E.g., the sections could become: Function Don’t Compose Well Functions Don’t Connect to Data Stores Well Otherwise Unnessary Queues are Needed Based on this article, it makes it sound like current serverless isn’t a very widely useful tool.
- scarface74 8y agoThere is more to Serverless than just APIs. For event based, eventually consistent workflows where you don’t need real time responses like working with queues, stream processing, etc. it’s great. But, I think it’s used a lot more than necessary for end user facing APIs they need fast responses.
- bazza451 8y agoI’m not going to fault him on the RDBMS comment, they scale shockingly due to the aforementioned connections comment but dynamodb is really not a replacement for a traditional RDBMS. But I’ve seen so many people get themselves into trouble by using dynamodb as a silver bullet and not thinking heavily about the design of their data. If you try and use RDBMS patterns (PK and FK relationships) there’s a whole heap of pain due to atomicity only being on one record in one table. Some times you can’t escape these patterns as well as it is the best fit for data or you require strong consistency of your model (e.g. financial). They released dynamodb transactions (for Java only) but from what I’ve seen that just increases the complexity of solutions, you end up with more reads/writes per call and a heap of other stuff surrounding your data. Serverless story currently for stuff like this really has drawbacks
- aggre 8y agoServer-less functions will be thinner when launched new managed services like AppSync.