4 ms·

I wonder what the difference between managed serverless and having an EC2 instance w/ Firecracker loaded up on it would be. I've used Lambda and I fear mainten

I wonder what the difference between managed serverless and having an EC2 instance w/ Firecracker loaded up on it would be. I've used Lambda and I fear maintenance burdens due to the underlying environments migrating; they'll keep the runtime for a running function for you but if it's old I don't think you can update that without having to upgrade runtimes. Also you get billed in 50ms increments at minimum, and so beyond that there's little incentive to make a function faster / a lot of incentive to stuff more logic into a function and make it less composable. I think I'd rather define an AMI w/ Firecracker using Packer, and stand up a box with Terraform, and write logs to CloudWatch, and have a much more predictable billing cycle / greater control.

sokoloff 6y ago
If your load is within a reasonably small range, Lambda is quite expensive on a per-computation basis. But, if your load varies wildly (and especially unpredictably), you can spend less time worrying about that than on a build-similar on EC2 solution.
- yingw787 6y ago
  I'm using Lambda@Edge for static site auth, so I just need to pass some HTTP headers, and it still seems a bit overkill. I really like the ability to integrate with a CDN, I don't think I want to move away from that to something like nginx that could fall over and be more expensive, but being billed for every signin I do is a bit nerve-wracking. Maybe it's just new and I don't have enough experience with it yet.
Dunedan 6y ago
> I've used Lambda and I fear maintenance burdens due to the underlying environments migrating; they'll keep the runtime for a running function for you but if it's old I don't think you can update that without having to upgrade runtimes. AWS supports runtimes as long as there are upstream support security updates available [1]. If there are no security updates anymore, it's a good idea anyway to update to a newer version. If course you don't have to do that if you host it on your own, but it's a good idea nevertheless. Also mind that most of the runtimes AWS deprecated already are old NodeJS versions. That's apparently because NodeJS only offers 30 months of support for their LTS versions [2]. So choosing a language which offers longer support (for example Python which offers 5 years of support [3]), might be a better choice if you don't want to update your software regularly. Also you could still provide your own runtime, which wouldn't be deprecated at all via custom runtimes [4]. > Also you get billed in 50ms increments at minimum, and so beyond that there's little incentive to make a function faster / a lot of incentive to stuff more logic into a function and make it less composable. You get billed in 100ms increments [5]. While I'd like to see smaller increments as well, I encountered more money thrown out of the window by overprovisioning memory (and therefore CPU) for AWS Lambda functions, than you could possibly save by optimizing for billable duration. [1]: https://docs.aws.amazon.com/lambda/latest/dg/runtime-support-policy.html https://docs.aws.amazon.com/lambda/latest/dg/runtime-support... [2]: https://nodejs.org/en/about/releases/ https://nodejs.org/en/about/releases/ [3]: https://devguide.python.org/#status-of-python-branches https://devguide.python.org/#status-of-python-branches [4]: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom... [5]: https://aws.amazon.com/lambda/pricing/ https://aws.amazon.com/lambda/pricing/
- yingw787 6y ago
  Hmm, you make some good points w.r.t. the custom runtime. I do kind of wish that AWS would maintain its own LTS releases of things like node.js, but I guess it's difficult to say whether that would require source mutation from the user. I do wish there were more examples of Lambda functions, like a marketplace of some sort. I'm using NodeJS only because certain tutorials offer examples in node.js (like static site authentication): https://douglasduhaime.com/posts/s3-lambda-auth.html https://douglasduhaime.com/posts/s3-lambda-auth.html I actually logged into CloudWatch just to see that point on 50ms vs. 100ms and yeah you're right, it's 100 not 50. I'm pretty new to Lambda, and wonder whether the billing insights dashboard will give me the breakdown b/w duration and memory. Thanks!
  - Dunedan 6y ago
    If you want to figure out if you overprovisioned memory for your AWS Lambda functions: There is a pretty handy pre-defined query in CloudWatch Insights for that.
scarface74 6y ago
That sounds like more of a maintenance nightmare. Why go through the hassle instead of just using Docker/Fargate with either ECS or EKS? If you don’t want to use Lambda?
- yingw787 6y ago
  I've only used Docker w/ ECS and EC2 task definitions, but creating and draining instances is pretty time-consuming and I'm guessing not responsive enough for serverless-specific workloads. If you use Packer and lock down your AMI and atomically update it, I don't see maintenance as being that big of a hassle, but I don't have too much experience in this regards and I value optionality to a high degree. I guess this would also depend on how many microVMs you might be able to reliably stick on an EC2 instance.
asguy 6y ago
Rolling your own firecracker solution would put the control in your hands, but then aren't you limited to .metal EC2 instances (with all the cost increases that come with it)?
- yingw787 6y ago
  I don't get why you have to go with .metal EC2 instances only, can you elaborate?
  - asguy 6y ago
    Firecracker uses KVM to implement virtualization to launch containers inside "micro" VMs. As far as I know, Amazon only gives you virtualization extension access on bare metal instances.

sokoloff 6y ago

If your load is within a reasonably small range, Lambda is quite expensive on a per-computation basis. But, if your load varies wildly (and especially unpredictably), you can spend less time worrying about that than on a build-similar on EC2 solution.

yingw787 6y ago
I'm using Lambda@Edge for static site auth, so I just need to pass some HTTP headers, and it still seems a bit overkill. I really like the ability to integrate with a CDN, I don't think I want to move away from that to something like nginx that could fall over and be more expensive, but being billed for every signin I do is a bit nerve-wracking. Maybe it's just new and I don't have enough experience with it yet.

Dunedan 6y ago

> I've used Lambda and I fear maintenance burdens due to the underlying environments migrating; they'll keep the runtime for a running function for you but if it's old I don't think you can update that without having to upgrade runtimes. AWS supports runtimes as long as there are upstream support security updates available [1]. If there are no security updates anymore, it's a good idea anyway to update to a newer version. If course you don't have to do that if you host it on your own, but it's a good idea nevertheless. Also mind that most of the runtimes AWS deprecated already are old NodeJS versions. That's apparently because NodeJS only offers 30 months of support for their LTS versions [2]. So choosing a language which offers longer support (for example Python which offers 5 years of support [3]), might be a better choice if you don't want to update your software regularly. Also you could still provide your own runtime, which wouldn't be deprecated at all via custom runtimes [4]. > Also you get billed in 50ms increments at minimum, and so beyond that there's little incentive to make a function faster / a lot of incentive to stuff more logic into a function and make it less composable. You get billed in 100ms increments [5]. While I'd like to see smaller increments as well, I encountered more money thrown out of the window by overprovisioning memory (and therefore CPU) for AWS Lambda functions, than you could possibly save by optimizing for billable duration. [1]: https://docs.aws.amazon.com/lambda/latest/dg/runtime-support-policy.html https://docs.aws.amazon.com/lambda/latest/dg/runtime-support... [2]: https://nodejs.org/en/about/releases/ https://nodejs.org/en/about/releases/ [3]: https://devguide.python.org/#status-of-python-branches https://devguide.python.org/#status-of-python-branches [4]: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom... [5]: https://aws.amazon.com/lambda/pricing/ https://aws.amazon.com/lambda/pricing/

yingw787 6y ago
Hmm, you make some good points w.r.t. the custom runtime. I do kind of wish that AWS would maintain its own LTS releases of things like node.js, but I guess it's difficult to say whether that would require source mutation from the user. I do wish there were more examples of Lambda functions, like a marketplace of some sort. I'm using NodeJS only because certain tutorials offer examples in node.js (like static site authentication): https://douglasduhaime.com/posts/s3-lambda-auth.html https://douglasduhaime.com/posts/s3-lambda-auth.html I actually logged into CloudWatch just to see that point on 50ms vs. 100ms and yeah you're right, it's 100 not 50. I'm pretty new to Lambda, and wonder whether the billing insights dashboard will give me the breakdown b/w duration and memory. Thanks!
- Dunedan 6y ago
  If you want to figure out if you overprovisioned memory for your AWS Lambda functions: There is a pretty handy pre-defined query in CloudWatch Insights for that.

scarface74 6y ago

That sounds like more of a maintenance nightmare. Why go through the hassle instead of just using Docker/Fargate with either ECS or EKS? If you don’t want to use Lambda?

yingw787 6y ago
I've only used Docker w/ ECS and EC2 task definitions, but creating and draining instances is pretty time-consuming and I'm guessing not responsive enough for serverless-specific workloads. If you use Packer and lock down your AMI and atomically update it, I don't see maintenance as being that big of a hassle, but I don't have too much experience in this regards and I value optionality to a high degree. I guess this would also depend on how many microVMs you might be able to reliably stick on an EC2 instance.

asguy 6y ago

Rolling your own firecracker solution would put the control in your hands, but then aren't you limited to .metal EC2 instances (with all the cost increases that come with it)?

yingw787 6y ago
I don't get why you have to go with .metal EC2 instances only, can you elaborate?
- asguy 6y ago
  Firecracker uses KVM to implement virtualization to launch containers inside "micro" VMs. As far as I know, Amazon only gives you virtualization extension access on bare metal instances.