3 ms·

Sorry this is going to be slightly tangential to the primary topic of this post. But does anyone work on backends that heavily utilize a serverless platform? To

by frfl 3y ago

Sorry this is going to be slightly tangential to the primary topic of this post. But does anyone work on backends that heavily utilize a serverless platform? To be more specific, backend (micro)services that embrace serverless fully (for example AWS Lambda, Dynamodb, StepFunctions, etc) and minimal usage of any ec2/container based deployments in the architecture? And to be clear, I'm not talking about some small bot or single internal app running on a serverless platform, rather an entire complex core business product or service that embraces serverless architecture -- think Amazon Prime previously using AWS Lambda type stuff. I'd love to know people's (objective if possible) thoughts, experiences and opinions on such an approach.

foofie 3y ago
> But does anyone work on backends that heavily utilize a serverless platform? As far as I know, that scenario is not in the decision tree for when to go with function-as-a-service offerings. FaaS is for things like glue code for event handlers, low-traffic services like API Gateway/NLB calling Lambdas, and expensive tasks that can be offloaded from a worker thread to a Lambda. When traffic going into a FaaS-based service starts to go up, the cost of running Lambdas stops being competitive and you're better off just having a dedicated service handle the work. Nevertheless, the main selling point of FaaS is not performance, throughput, or even developer convenience. The main selling point is hardware utilization. If your dev teams deploy FaaS stuff instead of dedicated services, the company doesn't need to provision anything to get them to run. The functions are just deployed in the sea of FaaS things they have been running, and they can live with high hardware utilization just fine.
- frfl 3y ago
  I ask because it's relevant to me, but not in my control. And still early stages and haven't hit that "When traffic going into a FaaS-basdd service starts to go up, the cost of running Lambdas stops being competitive" stage yet -- if what you're describing will be applicable and true in my situation. Sorry I can't go into specifics as it wouldn't be wise for me to discuss work stuff in more detail. So that's good to know. Do you have any references, in terms of further reading material, for the cross over point for when it's stops being competitive? I highly doubt $cloud_provider is going to give you such a analysis for obvious financial reasons.
  - foofie 3y ago
    > Do you have any references, in terms of further reading material, for the cross over point for when it's stops being competitive? I'm afraid I don't. The most objective cutoff point I can come up with is based purely on cost. Meaning, cloud providers like AWS might offer a free tier for API Gateway and Lambdas which makes them really competitive vs running a container or launching a VM when traffic is low. However, once your traffic starts to go beyond the free tier (IIRC per month it's 1million requests for API gateway and 400,000 GB/s for lambdas) then operational costs starts to overtake the cost of running a traditional service. How many requests your service handles on all endpoints and how much computational budget each lambda spends is a trait of your own service, and reflects low-level decision such as how you designed your API and what runtime did you chose for your lambdas. Basically this boils down to the FaaS solution eventually leading the cloud provider to charge you per each request, while your traditional service running either in a container or on a VM having a fixed cost. After some math you'll arrive at s traffic volume that can represent the tipping point between FaaS and traditional services. Personally, my subjective cutoff point is when FaaS costs about 30% of traditional setvices. Once your FaaS cost starts to grow, you're tempted to micro-optimize your API and your service to try to keep it's running cost justifiably low. That's when you're starting to be tempted to put together a sucky service as a cost tradeoff. When that time comes, it's better to bite the bullet and just migrate away from FaaS.
icedchai 3y ago
I've worked on low traffic apps where the entire backend API is running in Lambda. Would I do it again? Probably not. Lambda has a bunch of weird limitations (like request and response sizes) which means it's not suitable for all use cases. Additionally, you should avoid coding directly to the Lambda API. You want something to abstract that and make it look like a "normal" app. Otherwise, you wind up with a bunch of stuff you can't easily run locally.
mhotchen 3y ago
Yup. Entire core business product for a succeeding startup, though it's a small team of contributors (<10), and a much smaller platform team. Serverless backend started in 2018. Been a blessing in many regards, but it has its warts (often related to how new this architecture is, and of course we've made our own mistakes along the way). I really like the model of functions decoupled through events. Big fan of that. It's very flexible and iterative. Keep that as your focus and it's great. Be careful of duplicating config, look for ways to compose/reuse (duh, but definitely a lesson learnt) and same with CI, structure your project so it can use something off-the-shelf like serverless-compose. Definitely monorepo/monolith it, I'd be losing my mind with 100-150 repos/"microservices" with a team this size. If starting now I'd maybe look at SST framework[0] because redeploying every change during development gets old fast I couldn't go back to any other way to be honest, for cloud-heavy backends at least. By far the most productive I've ever been Definitely has its warts though, it's not all roses. [0] http://sst.dev http://sst.dev
- frfl 3y ago
  Are you using abstraction frameworks/libraries (eg https://www.serverless.com/framework https://www.serverless.com/framework) or using AWS/Azure/GCP directly?
  - mhotchen 3y ago
    We use serverless, quite happy with it. TBH I mentioned SST in my first comment but I'm not super experienced with it, and I'm reasonably happy with serverless. I'd definitely be careful in my own decision in trying out SST so I'd recommend doing the same for anyone who is taking my suggestion seriously
  - mhotchen 3y ago
    Oh and we only deploy functions and their IAM permissions with serverless. All other AWS resources are managed by terraform. I think this was wise
gary17the 3y ago
I could be wrong, but from my experience, serverless has only one crucial advantage over serverful: the ability to instantly scale up on a moment's notice. You deploy some code to an established cloud provider, the cloud infrastructure replicates the code across hundreds of compute instances in a hundred geographical locations (possibly utilizing a cloud data store, synchronized internally across all the compute nodes used) and suddenly there is no longer a single performance-sensitive "failure point". Your product can get on the front page of HN and survive. When it comes to serverful, you have to provision (or perhaps "over-provision") resources ahead of time. If you can estimate your CPU/memory/disk/network needs beforehand, you can just as well rent an appropriately powerful, "serverful" server to achieve your goal. If you assume (figure out) a particular desired number of API requests per second that you need to handle, analyze and benchmark a particular set of hardware resources that can likely handle such throughput and also calculate a per-request -based price of the same throughput with a serverless provider, you will probably find that serverless is at least twice as expensive as serverful.
bob1029 3y ago
Our go-forward architecture runs everything on top of Azure Functions. We are using HttpTrigger with basic SSR web forms to deliver the solution. There is no "back-end". The functions directly serve the customer traffic (text/html get & post) and talk to whatever SQL database / external systems are required. We are finding that this is a much easier operational model in a B2B/SaaS consulting setting wherein our customers (banks & credit unions) must retain legal ownership over their data. Setting up a VM/container circus per customer and expecting them to keep it healthy is not something we are prepared to continue beyond client #5 or so. There is zero reality wherein we could maintain some kind of multi-tenant system. After targeting this stack seriously for about a year, I am having a hard time coming up with a reason I wouldn't use it for everything. Not having to think about VMs and weird crusty infra management code has freed up an incredible amount of bandwidth for our development team.
mattchamb 3y ago
The company I work at is using step functions heavily and I hate it. Instead of an if statement they make it a new step with the conditional in json, so following the code required you to jump around between files. It has also been built by contractors who have no incentives to make it run locally or be easy to manage in production, but that isn't specific to step functions, and more due to poor leadership.
- Yhippa 3y ago
  I never thought about it that way. Sounds like step functions could end up "spaghetti code on steroids" if you do it poorly.
  - frfl 3y ago
    Imagine using JSON as a programming language. And all the implications of that. AWS does provide a VS Code and web renderer for the JSON to visually see what the flow of the step function looks like, but this is just a small improvement overall. If you split up your step functions into multiple, where one step function can call another, also invoke lambdas and other things, then you can imagine you lose all benefits of a modern code editor/IDE and you're manually lookup which file to jump to in order to find the next step in the execution path. To be fair AWS does provide a "CDK" which lets you write Python code that gets converted into their JSON DSL -- or something along those lines. But I haven't used that, only direct JSON DSL "code" to write step functions.
helen___keller 3y ago
I’ve worked on a product where half the backend (basically, the control plane but not the data plane) was completely serverless: lambda, DDB, step functions, api gateway I think it was generally convenient as being very low maintenance. The only availability canary alarms are when the underlying services go down, which is almost never. Debugging and running locally are the weak points. Eliminating an entire class of things to think about (container health, scale in and out, and so on) is very nice. Since it’s a control plane, tps and costs were low.

by frfl 3y ago

foofie 3y ago

> But does anyone work on backends that heavily utilize a serverless platform? As far as I know, that scenario is not in the decision tree for when to go with function-as-a-service offerings. FaaS is for things like glue code for event handlers, low-traffic services like API Gateway/NLB calling Lambdas, and expensive tasks that can be offloaded from a worker thread to a Lambda. When traffic going into a FaaS-based service starts to go up, the cost of running Lambdas stops being competitive and you're better off just having a dedicated service handle the work. Nevertheless, the main selling point of FaaS is not performance, throughput, or even developer convenience. The main selling point is hardware utilization. If your dev teams deploy FaaS stuff instead of dedicated services, the company doesn't need to provision anything to get them to run. The functions are just deployed in the sea of FaaS things they have been running, and they can live with high hardware utilization just fine.

frfl 3y ago
I ask because it's relevant to me, but not in my control. And still early stages and haven't hit that "When traffic going into a FaaS-basdd service starts to go up, the cost of running Lambdas stops being competitive" stage yet -- if what you're describing will be applicable and true in my situation. Sorry I can't go into specifics as it wouldn't be wise for me to discuss work stuff in more detail. So that's good to know. Do you have any references, in terms of further reading material, for the cross over point for when it's stops being competitive? I highly doubt $cloud_provider is going to give you such a analysis for obvious financial reasons.
- foofie 3y ago
  > Do you have any references, in terms of further reading material, for the cross over point for when it's stops being competitive? I'm afraid I don't. The most objective cutoff point I can come up with is based purely on cost. Meaning, cloud providers like AWS might offer a free tier for API Gateway and Lambdas which makes them really competitive vs running a container or launching a VM when traffic is low. However, once your traffic starts to go beyond the free tier (IIRC per month it's 1million requests for API gateway and 400,000 GB/s for lambdas) then operational costs starts to overtake the cost of running a traditional service. How many requests your service handles on all endpoints and how much computational budget each lambda spends is a trait of your own service, and reflects low-level decision such as how you designed your API and what runtime did you chose for your lambdas. Basically this boils down to the FaaS solution eventually leading the cloud provider to charge you per each request, while your traditional service running either in a container or on a VM having a fixed cost. After some math you'll arrive at s traffic volume that can represent the tipping point between FaaS and traditional services. Personally, my subjective cutoff point is when FaaS costs about 30% of traditional setvices. Once your FaaS cost starts to grow, you're tempted to micro-optimize your API and your service to try to keep it's running cost justifiably low. That's when you're starting to be tempted to put together a sucky service as a cost tradeoff. When that time comes, it's better to bite the bullet and just migrate away from FaaS.

icedchai 3y ago

I've worked on low traffic apps where the entire backend API is running in Lambda. Would I do it again? Probably not. Lambda has a bunch of weird limitations (like request and response sizes) which means it's not suitable for all use cases. Additionally, you should avoid coding directly to the Lambda API. You want something to abstract that and make it look like a "normal" app. Otherwise, you wind up with a bunch of stuff you can't easily run locally.

mhotchen 3y ago

Yup. Entire core business product for a succeeding startup, though it's a small team of contributors (<10), and a much smaller platform team. Serverless backend started in 2018. Been a blessing in many regards, but it has its warts (often related to how new this architecture is, and of course we've made our own mistakes along the way). I really like the model of functions decoupled through events. Big fan of that. It's very flexible and iterative. Keep that as your focus and it's great. Be careful of duplicating config, look for ways to compose/reuse (duh, but definitely a lesson learnt) and same with CI, structure your project so it can use something off-the-shelf like serverless-compose. Definitely monorepo/monolith it, I'd be losing my mind with 100-150 repos/"microservices" with a team this size. If starting now I'd maybe look at SST framework[0] because redeploying every change during development gets old fast I couldn't go back to any other way to be honest, for cloud-heavy backends at least. By far the most productive I've ever been Definitely has its warts though, it's not all roses. [0] http://sst.dev http://sst.dev

frfl 3y ago
Are you using abstraction frameworks/libraries (eg https://www.serverless.com/framework https://www.serverless.com/framework) or using AWS/Azure/GCP directly?
- mhotchen 3y ago
  We use serverless, quite happy with it. TBH I mentioned SST in my first comment but I'm not super experienced with it, and I'm reasonably happy with serverless. I'd definitely be careful in my own decision in trying out SST so I'd recommend doing the same for anyone who is taking my suggestion seriously
- mhotchen 3y ago
  Oh and we only deploy functions and their IAM permissions with serverless. All other AWS resources are managed by terraform. I think this was wise

gary17the 3y ago

I could be wrong, but from my experience, serverless has only one crucial advantage over serverful: the ability to instantly scale up on a moment's notice. You deploy some code to an established cloud provider, the cloud infrastructure replicates the code across hundreds of compute instances in a hundred geographical locations (possibly utilizing a cloud data store, synchronized internally across all the compute nodes used) and suddenly there is no longer a single performance-sensitive "failure point". Your product can get on the front page of HN and survive. When it comes to serverful, you have to provision (or perhaps "over-provision") resources ahead of time. If you can estimate your CPU/memory/disk/network needs beforehand, you can just as well rent an appropriately powerful, "serverful" server to achieve your goal. If you assume (figure out) a particular desired number of API requests per second that you need to handle, analyze and benchmark a particular set of hardware resources that can likely handle such throughput and also calculate a per-request -based price of the same throughput with a serverless provider, you will probably find that serverless is at least twice as expensive as serverful.

bob1029 3y ago

Our go-forward architecture runs everything on top of Azure Functions. We are using HttpTrigger with basic SSR web forms to deliver the solution. There is no "back-end". The functions directly serve the customer traffic (text/html get & post) and talk to whatever SQL database / external systems are required. We are finding that this is a much easier operational model in a B2B/SaaS consulting setting wherein our customers (banks & credit unions) must retain legal ownership over their data. Setting up a VM/container circus per customer and expecting them to keep it healthy is not something we are prepared to continue beyond client #5 or so. There is zero reality wherein we could maintain some kind of multi-tenant system. After targeting this stack seriously for about a year, I am having a hard time coming up with a reason I wouldn't use it for everything. Not having to think about VMs and weird crusty infra management code has freed up an incredible amount of bandwidth for our development team.

mattchamb 3y ago

The company I work at is using step functions heavily and I hate it. Instead of an if statement they make it a new step with the conditional in json, so following the code required you to jump around between files. It has also been built by contractors who have no incentives to make it run locally or be easy to manage in production, but that isn't specific to step functions, and more due to poor leadership.

Yhippa 3y ago
I never thought about it that way. Sounds like step functions could end up "spaghetti code on steroids" if you do it poorly.
- frfl 3y ago
  Imagine using JSON as a programming language. And all the implications of that. AWS does provide a VS Code and web renderer for the JSON to visually see what the flow of the step function looks like, but this is just a small improvement overall. If you split up your step functions into multiple, where one step function can call another, also invoke lambdas and other things, then you can imagine you lose all benefits of a modern code editor/IDE and you're manually lookup which file to jump to in order to find the next step in the execution path. To be fair AWS does provide a "CDK" which lets you write Python code that gets converted into their JSON DSL -- or something along those lines. But I haven't used that, only direct JSON DSL "code" to write step functions.

helen___keller 3y ago

I’ve worked on a product where half the backend (basically, the control plane but not the data plane) was completely serverless: lambda, DDB, step functions, api gateway I think it was generally convenient as being very low maintenance. The only availability canary alarms are when the underlying services go down, which is almost never. Debugging and running locally are the weak points. Eliminating an entire class of things to think about (container health, scale in and out, and so on) is very nice. Since it’s a control plane, tps and costs were low.