6 ms·

This is from the perspective of a small startup CTO: We've used AWS Lambda for about 4 years, and it's been so good and so cheap that I'm shifting literally ev

This is from the perspective of a small startup CTO: We've used AWS Lambda for about 4 years, and it's been so good and so cheap that I'm shifting literally everything (except Redis) to serverless. Also, GCP has a better serverless offering (Cloud Run, Spanner), so we're switching from AWS to GCP to take advantage of that. I bet we're going to see a massive cost reduction, but we'll see. Things I like about serverless (again, from the perspective of a very small startup, with 5 engineers, and me being the primary architect): * It's so liberating to not worry about EC2 servers and autoscale and container orchestration myself. All our Cloud Formation templates add up to around 3,000 lines, which maybe doesn't sound like a lot, but it's a lot. There are tons of little configuration things to worry about, and it adds up. (Not to mention the sheer amount of time it took to learn.) ECS Fargate takes care of some of this, but it doesn't autoscale based on demand or anything (not without settings things up yourself). (This is a big reason why I want to switch to GCP: Cloud Run is like Fargate in that it runs containers, but unlike Fargate it autoscales from 0 based on load.) * It's very cheap in practice, at least for loads like ours that respond to events: API services that sometimes see a lot of use and sometimes see very little use; queue consumers sometimes have a lot to do and sometimes have very little to do. AWS Lambda bills down to the milisecond in terms of resolution, and GCP Cloud Run/Cloud Funcitons bills down to the next 100 miliseconds. These are very fine resolutions and for us at least, we've seen costs be small. * For database serverless products (like DynamoDB for example), it's very liberating to never have to think "Hm, do we have enough CPU provisioned?" Things I don't like about serverless * Pushing source code sucks. Lambda will just one day decide your version of Python or whatever isn't good enough and force your customers to upgrade all their user-written code to the latest Python version. (But! Cloud Run supports containers, and so this won't be a problem.)

intrasight 4y ago
>GCP has a better serverless offering I am starting to evaluate AWS for GCP for serverless. What, in your opinion, makes GCP better? Is the comment in the context of containers or functions?
- adameasterling 4y ago
  I have limited time before my next meeting so I'll type real quick: GCP Cloud Run is like the best of both worlds between AWS ECS Fargate and AWS Lambda. (Yes, the comment is in the context of containers. Sort of.) * Like Fargate, Cloud Run hosts containers and takes care of figuring out where they actually live. Unlike Fargate, you don't have to say exactly how many containers you want running at once; GCP will automatically scale the # of containers up and down based on HTTP load and will scale down to 0. This should make Cloud Run cheaper than Fargate. (If you want to hook up Fargate to a webserver and you don't have autoscale figured out, you'll have to keep a lot of workers alive doing nothing.) * Like Lambda, Cloud Run bills by the amount of time spent processing at least one request. But unlike Lambda, Cloud Run lets one container handle more than one request at a time (it sucks to have to spin up a lot of Lambda invocations that do a bunch of IO). Web servers that are good at concurrency shine here. This should save money. * Cloud Run has more generous limits in many respects than Lambda. Cloud Run lets you set up SIGTERM hooks, so you can do some cleanup logic in your container (to e.g. write performance data to a timeseries table or whatever). That's Cloud Run. On the database side: GCP Firestore is very interesting and we're going to build a big feature around it. AWS has nothing like it. On the queue side of things: We're planning to build around GCP Cloud Tasks; We've more or less built Cloud Tasks ourselves using a mix of MySQL and AWS SQS (and it was hard and we haven't done a good job). I'd love to start a Discord or something to discuss these thoughts more. It's so hard to get good practical information for system architects/CTO types who just need to hammer stuff out.
  - idunno246 4y ago
    this falls in line with how i feel about cloud run, it really feels like a much better abstraction than ecs/gke or functions. it also is more similar to how most devs currently work, and local dev is the same. for non-api based traffic, like queues, it has some really weird quirks, like the autoscaler is problematic. but our experience with gcp in general has ranged from mediocre to bad, where the tech seems cool but things dont quite fit together
  - ManWith2Plans 4y ago
    A couple of points: 1. Cloud Run is more analogous to AWS App Runner than Fargate. 2. Cloud Run isn't a great analog to lambda. Lambda is built to host functions. Cloud Run is built to host applications. Lambda is more analogous to GCP Functions. 3. Cloud Tasks should probably be built with EventBridge + Lambda or EventBridge + StepFunctions or EventBridge + ECS. I don't profess to be a GCP expert so it's hard for me to make a judgement call on what's better. I can, however, say that most of this post ignores some of the real serverless power provided by AWS. AWS AppSync, AWS API Gateway, DynamoDB, CloudFront Functions, Lambda@Edge. It also makes comparisons that are not very fair.
Azkar 4y ago
AWS Lambda supports shipping your Lambdas as containers now. Very nice experience.
- apendleton 4y ago
  Was going to comment this same thing. We've been finding that the cold-start times are somewhat worse, but not disastrously so.
jedberg 4y ago
> I'm shifting literally everything (except Redis) to serverless A new company called Momento just launched that offers a serverless cache. Might be of interest to you. https://www.gomomento.com https://www.gomomento.com
tra3 4y ago
I just started working with Lambda, so my experience may just be the teething pains, but. I find the developer experience a bit jarring. My lambda is in python, but every time I want to test my code I have to "build" the lambda with the `sam build` tool and only then can I exercise it. It takes a non-trivial amount of time to build. The deployment story looks good though, with the `sam deploy`, but for now I can't get over the developer experience.
- coredog64 4y ago
  My recommendation is to invest in testing now while it’s easy. You can mock the incoming events and use something like moto for service mocks. That catches dumb mistakes sooner. Also, don’t put any logic in the handler itself unless it’s extremely trivial.
  - tra3 4y ago
    Sound advice, but that ship has sailed. This is an existing lambda that I'm refactoring so I need to be able to "go through the front door", as it were. Once I become more familiar with the code, I'll be able to test individual pieces are you're recommending.
willio58 4y ago
This has been my experience so far too. Serverless is amazing, it does require some shift in thought when it comes to backend architecture but once you get there it really does provide everything it promises. Less cost, less complexity (if done right), and the scalability is amazing. I agree it's not perfect but I highly recommend any backend or fullstack dev take a look at if you haven't already. Been working on a lambda, dynamodb, typescript react app for about 6 months for work and it's been just mind-melting how much money and complexity we've saved switching. I'm talking like 5x the cost drop and we can more easily onboard devs to the project because it's just simpler, no need for a devops hire honestly.
- hbrn 4y ago
  Can you expand a bit on complexity part? How exactly serverless reduces it? The only thing that comes to mind better isolation between parts of the app, but this could be achieved with any architecture if done right.
  - willio58 4y ago
    We don’t have to manage a server, that’s really the drop in complexity. We just write a function and it runs on a machine somewhere in the cloud. As we scale, AWS just handles it automatically. If the demand decreases, no problem. We just pay for what we use. Of course there are ways to handle scale with servers, but with serverless you barely have to think about it.
spiffytech 4y ago
Does your team do local dev? Every team I've known that adopted Lambda + DynamoDB (or equivalents) gave up on running their app locally, adding a lot of friction to the development process.
- birdstheword5 4y ago
  This is one of my concerns as well - apparently azure functions allows you to debug your function from within vscode. I see lots of issues with this in that 1. you are limited to vscode as your editor and 2. you can only interact with Azure resources in a "development environment" within Azure itself, i.e. no local copy of the database etc
- adameasterling 4y ago
  Within Lambda, yes, sucks. This is why container-based serverless is so much more exciting. (Which Cloud Run offers.) I'm in the early stages of this rearchitecture but so far I've had no difficulty with local development.
- jeffybefffy519 4y ago
  I highly recommend using AWS's own Chalice library as it makes local dev _and_ deployment very easy. If you need more complex cases like deploying docker containers to a Lambda function, take a look at AWS's SAM library. Also supports local dev _and_ makes deployment easy (its essentially a wrapper around Cloudformation so its very powerful).
youens 4y ago
I agree with this line of thought. (Also from the perspective of a small startup CTO with ~10 developers mixed across golang, python, and react.) We used GCP at our previous startup (sold ) and ran our own K8S, when it was very new (2015). There were lots of pains in those days. So when we started our current startup in 2018, we started with App Engine (flexible, which supports containers). This was fine, but lots of drawbacks. After a year or two we ended up back on K8S, using GCP's GKE (managed K8S). Our team is pretty good with K8S, so it was fine. But regardless, the little stuff adds up. Fast forward to about 6 months ago. We had used GCP's Cloud Run off and on for little stuff, and it kept getting better. One day someone asked the question why we shouldn't just use it for everything. Everyone was a bit defensive, but we kind of stared at each other and couldn't think of great reasons (for our use case), so we tried it. Our setup consists of a primary API service (Golang), and a dozen or so smaller microservices, mainly in Python. We even moved most of our React apps to cloud run. 6 months in, and I can't really say anything bad. We turn off scale to 0 for the services where it matters. It scales up quickly to loads, zero down time over 6 months, no troubleshooting (so. much. time. saved.), super easy to deploy, swap traffic between versions, etc. I'm not saying it a silver bullet, nor that it's perfect for everyone... but I couldn't say enough good things about _container-based_ serverless like Cloud Run. That said, breaking big systems down to the function level (Lambda, GCP Cloud Functions, etc) sounds like a nightmare to me. I'm sure there are ways, but that's a different ballgame. We do use FaaS for some tasks. YMMV. Edit: Oh, and our hosting bill went from ~$5k a month to $500 a month (in part to other things, but primarily the lack of need for big node pools.)

intrasight 4y ago

>GCP has a better serverless offering I am starting to evaluate AWS for GCP for serverless. What, in your opinion, makes GCP better? Is the comment in the context of containers or functions?

adameasterling 4y ago
I have limited time before my next meeting so I'll type real quick: GCP Cloud Run is like the best of both worlds between AWS ECS Fargate and AWS Lambda. (Yes, the comment is in the context of containers. Sort of.) * Like Fargate, Cloud Run hosts containers and takes care of figuring out where they actually live. Unlike Fargate, you don't have to say exactly how many containers you want running at once; GCP will automatically scale the # of containers up and down based on HTTP load and will scale down to 0. This should make Cloud Run cheaper than Fargate. (If you want to hook up Fargate to a webserver and you don't have autoscale figured out, you'll have to keep a lot of workers alive doing nothing.) * Like Lambda, Cloud Run bills by the amount of time spent processing at least one request. But unlike Lambda, Cloud Run lets one container handle more than one request at a time (it sucks to have to spin up a lot of Lambda invocations that do a bunch of IO). Web servers that are good at concurrency shine here. This should save money. * Cloud Run has more generous limits in many respects than Lambda. Cloud Run lets you set up SIGTERM hooks, so you can do some cleanup logic in your container (to e.g. write performance data to a timeseries table or whatever). That's Cloud Run. On the database side: GCP Firestore is very interesting and we're going to build a big feature around it. AWS has nothing like it. On the queue side of things: We're planning to build around GCP Cloud Tasks; We've more or less built Cloud Tasks ourselves using a mix of MySQL and AWS SQS (and it was hard and we haven't done a good job). I'd love to start a Discord or something to discuss these thoughts more. It's so hard to get good practical information for system architects/CTO types who just need to hammer stuff out.
- idunno246 4y ago
  this falls in line with how i feel about cloud run, it really feels like a much better abstraction than ecs/gke or functions. it also is more similar to how most devs currently work, and local dev is the same. for non-api based traffic, like queues, it has some really weird quirks, like the autoscaler is problematic. but our experience with gcp in general has ranged from mediocre to bad, where the tech seems cool but things dont quite fit together
- ManWith2Plans 4y ago
  A couple of points: 1. Cloud Run is more analogous to AWS App Runner than Fargate. 2. Cloud Run isn't a great analog to lambda. Lambda is built to host functions. Cloud Run is built to host applications. Lambda is more analogous to GCP Functions. 3. Cloud Tasks should probably be built with EventBridge + Lambda or EventBridge + StepFunctions or EventBridge + ECS. I don't profess to be a GCP expert so it's hard for me to make a judgement call on what's better. I can, however, say that most of this post ignores some of the real serverless power provided by AWS. AWS AppSync, AWS API Gateway, DynamoDB, CloudFront Functions, Lambda@Edge. It also makes comparisons that are not very fair.

Azkar 4y ago

AWS Lambda supports shipping your Lambdas as containers now. Very nice experience.

apendleton 4y ago
Was going to comment this same thing. We've been finding that the cold-start times are somewhat worse, but not disastrously so.

jedberg 4y ago

> I'm shifting literally everything (except Redis) to serverless A new company called Momento just launched that offers a serverless cache. Might be of interest to you. https://www.gomomento.com https://www.gomomento.com

tra3 4y ago

I just started working with Lambda, so my experience may just be the teething pains, but. I find the developer experience a bit jarring. My lambda is in python, but every time I want to test my code I have to "build" the lambda with the `sam build` tool and only then can I exercise it. It takes a non-trivial amount of time to build. The deployment story looks good though, with the `sam deploy`, but for now I can't get over the developer experience.

coredog64 4y ago
My recommendation is to invest in testing now while it’s easy. You can mock the incoming events and use something like moto for service mocks. That catches dumb mistakes sooner. Also, don’t put any logic in the handler itself unless it’s extremely trivial.
- tra3 4y ago
  Sound advice, but that ship has sailed. This is an existing lambda that I'm refactoring so I need to be able to "go through the front door", as it were. Once I become more familiar with the code, I'll be able to test individual pieces are you're recommending.

willio58 4y ago

This has been my experience so far too. Serverless is amazing, it does require some shift in thought when it comes to backend architecture but once you get there it really does provide everything it promises. Less cost, less complexity (if done right), and the scalability is amazing. I agree it's not perfect but I highly recommend any backend or fullstack dev take a look at if you haven't already. Been working on a lambda, dynamodb, typescript react app for about 6 months for work and it's been just mind-melting how much money and complexity we've saved switching. I'm talking like 5x the cost drop and we can more easily onboard devs to the project because it's just simpler, no need for a devops hire honestly.

hbrn 4y ago
Can you expand a bit on complexity part? How exactly serverless reduces it? The only thing that comes to mind better isolation between parts of the app, but this could be achieved with any architecture if done right.
- willio58 4y ago
  We don’t have to manage a server, that’s really the drop in complexity. We just write a function and it runs on a machine somewhere in the cloud. As we scale, AWS just handles it automatically. If the demand decreases, no problem. We just pay for what we use. Of course there are ways to handle scale with servers, but with serverless you barely have to think about it.

spiffytech 4y ago

Does your team do local dev? Every team I've known that adopted Lambda + DynamoDB (or equivalents) gave up on running their app locally, adding a lot of friction to the development process.

birdstheword5 4y ago
This is one of my concerns as well - apparently azure functions allows you to debug your function from within vscode. I see lots of issues with this in that 1. you are limited to vscode as your editor and 2. you can only interact with Azure resources in a "development environment" within Azure itself, i.e. no local copy of the database etc
adameasterling 4y ago
Within Lambda, yes, sucks. This is why container-based serverless is so much more exciting. (Which Cloud Run offers.) I'm in the early stages of this rearchitecture but so far I've had no difficulty with local development.
jeffybefffy519 4y ago
I highly recommend using AWS's own Chalice library as it makes local dev _and_ deployment very easy. If you need more complex cases like deploying docker containers to a Lambda function, take a look at AWS's SAM library. Also supports local dev _and_ makes deployment easy (its essentially a wrapper around Cloudformation so its very powerful).

youens 4y ago

I agree with this line of thought. (Also from the perspective of a small startup CTO with ~10 developers mixed across golang, python, and react.) We used GCP at our previous startup (sold ) and ran our own K8S, when it was very new (2015). There were lots of pains in those days. So when we started our current startup in 2018, we started with App Engine (flexible, which supports containers). This was fine, but lots of drawbacks. After a year or two we ended up back on K8S, using GCP's GKE (managed K8S). Our team is pretty good with K8S, so it was fine. But regardless, the little stuff adds up. Fast forward to about 6 months ago. We had used GCP's Cloud Run off and on for little stuff, and it kept getting better. One day someone asked the question why we shouldn't just use it for everything. Everyone was a bit defensive, but we kind of stared at each other and couldn't think of great reasons (for our use case), so we tried it. Our setup consists of a primary API service (Golang), and a dozen or so smaller microservices, mainly in Python. We even moved most of our React apps to cloud run. 6 months in, and I can't really say anything bad. We turn off scale to 0 for the services where it matters. It scales up quickly to loads, zero down time over 6 months, no troubleshooting (so. much. time. saved.), super easy to deploy, swap traffic between versions, etc. I'm not saying it a silver bullet, nor that it's perfect for everyone... but I couldn't say enough good things about _container-based_ serverless like Cloud Run. That said, breaking big systems down to the function level (Lambda, GCP Cloud Functions, etc) sounds like a nightmare to me. I'm sure there are ways, but that's a different ballgame. We do use FaaS for some tasks. YMMV. Edit: Oh, and our hosting bill went from ~$5k a month to $500 a month (in part to other things, but primarily the lack of need for big node pools.)