5 ms·

I'm terribly skeptical of AWS Lambda as the engine of a main-line REST API. There seems to be no way to avoid the cold-start time: not only when your function i

I'm terribly skeptical of AWS Lambda as the engine of a main-line REST API. There seems to be no way to avoid the cold-start time: not only when your function is first called from a dormant state, but also when request load rises, bringing additional workers online. (A timer that constantly pings your service only protects against the first, and I hope we all agree is a dirty hack.) I'm hoping someone will correct me if I'm missing something. Otherwise it seems like Lambda isn't really suited to use-cases where random latency spikes would degrade user experience.

scrollaway 9y ago
Fun fact, on the lowest memory available, cold start times are really bad in Python when you have a lot of libraries that need to be imported. The Lambda runtime is exceedingly slow at that on the 128MB tier. This is in fact the case for Django apps, even in their default state. Now here's where the fun fact starts: If a function times out during cold start, it hasn't successfully been warmed. That means upon its next trigger, it will coldstart again. Now lets say you deploy a tiny little function over wsgi. You see in your metrics that it takes an avg of 200ms... now, you're a smart dev, you set its global timeout to 3 seconds down from the default of 30 seconds because you don't want to get billed more than necessary. But as it turns out, the cold start takes on average 10 seconds. Your function now never successfully completes.
- smashedtoatoms 9y ago
  Sure, but why would you do that? If you’re running Django, set up your ec2 instance, and run Django. If you’re running lambda, don’t spin up an entire framework in it to run for individual requests. That basically defeats the purpose. You’re not running a tiny little function, as you said. Your running an entire web framework that’s running one tiny function. Instead of doing that, write an actual simple function that does the one thing it needs to do. It will take 200-300 ms to spin up and run the first time, as well as on requests when it scales up, but otherwise will run in 2ms. Keep it really simple and stateless. If some part of that doesn’t work for your use case, then don’t use lambda. It was not meant to run an entire framework in. If you’re reaching for an entire framework to run a simple function in a lambda, I’d bail on one or the other. Don’t use a framework, or don’t use lambda.
  - scrollaway 9y ago
    I mostly agree, but sometimes it's simply convenient to use the regular Django wsgi routines to serve parts of a Django app. The alternative there if you want to serve stuff behind URLs is using the Api Gateway and that is atrocious to work with. Also, whenever there's a discussion about lambda, people talk about "lock in" and the api gateway is a much bigger lock-in than lambda. Example: I have an API in Django which uses DRF, served on a classic web server instance behind load balancers etc. Parts of that API, which have the same authentication/business logic/etc, are much more suited to lambda, I'm sure it's not hard to imagine :)
    - smashedtoatoms 9y ago
      That’s fair. Api Gateway is definitely the lock-in as well.
      - _pdp_ 9y ago
        Everything is a lock-in. :) Lambda has a straightforward interface so wrapping your functions in something else should be easy enough.
        k__ 9y ago
        Also, Lambda is just the glue code. You have to use many other AWS stuff that really locks you in deep, even if you functionallity could "theoretically" be ported to different providers.
        _pdp_ 9y ago
        If there is a real need to port your infrastructure you can do it in small steps. Using single cloud provider is also not smart. If I need to do huge data processing I will never do it with AWS although this is where the rest of the infrastructure is as Google has better solutions. Just pick whatever is best.
- mcintyre1994 9y ago
  Azure takes ~30s as soon as you add an import. Admittedly their Python is still marked experimental though.
smashedtoatoms 9y ago
You’re spot on. It’s really not good for any scenario where an occasional random long response time is a deal breaker. It gets particularly crappy when you have lambdas dependent on other lambdas. That being said, if an occasional long request is ok, it’s pretty great. It just depends on what you’re doing with it.
jacques_chester 9y ago
The advantage and disadvantage of lambda is that it decouples your economics from AWS's (or Azure's or GCP's) economics. That is, the cost of idle time is shifted from your account to their account. This sets up a bit of a race: Amazon's goal is to have as few hidden warm instances for as little time as possible. The user's goal is to have as many hidden warm instances as possible for as long as possible. This is where you start to see pinging to provoke AWS into keeping a warm copy. As time goes on they will have a growing incentive to prevent tricks like these from working. At the moment, on the economics, hosted FaaSes will always have this tension. Self-hosted FaaSes open up the possibility of setting your own policy for trading latency against idle cost. Disclosure: I am part of a team working on Project Riff, a FaaS.
- Turbots 9y ago
  I saw mark fishers riff demo at SpringOne and really liked it, just set up riff on my Mac with kubernetes on docker, I'm working on a fun demo right now, using kafka as a buffer is quite neat! Excited to see what this project will evolve into
- merb 9y ago
  > Self-hosted FaaSes open up the possibility of setting your own policy for trading latency against idle cost if you already have a K8s cluster it's probably cheaper to just create your application regulary and use it like that. No need to use a FaaS which complicates your whole setup for no additional benefit.
  - jacques_chester 9y ago
    Self-hosted isn't free either. You're still paying for resources when they're idle. The difference is that you can decide on whether to scale down to 1 or not. As it happens Riff and most other FaaS projects are built on kubernetes.
    - merb 9y ago
      > Self-hosted isn't free either. You're still paying for resources when they're idle. I still have these resources idle, when I use Riff.

scrollaway 9y ago

Fun fact, on the lowest memory available, cold start times are really bad in Python when you have a lot of libraries that need to be imported. The Lambda runtime is exceedingly slow at that on the 128MB tier. This is in fact the case for Django apps, even in their default state. Now here's where the fun fact starts: If a function times out during cold start, it hasn't successfully been warmed. That means upon its next trigger, it will coldstart again. Now lets say you deploy a tiny little function over wsgi. You see in your metrics that it takes an avg of 200ms... now, you're a smart dev, you set its global timeout to 3 seconds down from the default of 30 seconds because you don't want to get billed more than necessary. But as it turns out, the cold start takes on average 10 seconds. Your function now never successfully completes.

smashedtoatoms 9y ago
Sure, but why would you do that? If you’re running Django, set up your ec2 instance, and run Django. If you’re running lambda, don’t spin up an entire framework in it to run for individual requests. That basically defeats the purpose. You’re not running a tiny little function, as you said. Your running an entire web framework that’s running one tiny function. Instead of doing that, write an actual simple function that does the one thing it needs to do. It will take 200-300 ms to spin up and run the first time, as well as on requests when it scales up, but otherwise will run in 2ms. Keep it really simple and stateless. If some part of that doesn’t work for your use case, then don’t use lambda. It was not meant to run an entire framework in. If you’re reaching for an entire framework to run a simple function in a lambda, I’d bail on one or the other. Don’t use a framework, or don’t use lambda.
- scrollaway 9y ago
  I mostly agree, but sometimes it's simply convenient to use the regular Django wsgi routines to serve parts of a Django app. The alternative there if you want to serve stuff behind URLs is using the Api Gateway and that is atrocious to work with. Also, whenever there's a discussion about lambda, people talk about "lock in" and the api gateway is a much bigger lock-in than lambda. Example: I have an API in Django which uses DRF, served on a classic web server instance behind load balancers etc. Parts of that API, which have the same authentication/business logic/etc, are much more suited to lambda, I'm sure it's not hard to imagine :)
  - smashedtoatoms 9y ago
    That’s fair. Api Gateway is definitely the lock-in as well.
    - _pdp_ 9y ago
      Everything is a lock-in. :) Lambda has a straightforward interface so wrapping your functions in something else should be easy enough.
      - k__ 9y ago
        Also, Lambda is just the glue code. You have to use many other AWS stuff that really locks you in deep, even if you functionallity could "theoretically" be ported to different providers.
        _pdp_ 9y ago
        If there is a real need to port your infrastructure you can do it in small steps. Using single cloud provider is also not smart. If I need to do huge data processing I will never do it with AWS although this is where the rest of the infrastructure is as Google has better solutions. Just pick whatever is best.
mcintyre1994 9y ago
Azure takes ~30s as soon as you add an import. Admittedly their Python is still marked experimental though.

smashedtoatoms 9y ago

You’re spot on. It’s really not good for any scenario where an occasional random long response time is a deal breaker. It gets particularly crappy when you have lambdas dependent on other lambdas. That being said, if an occasional long request is ok, it’s pretty great. It just depends on what you’re doing with it.

jacques_chester 9y ago

The advantage and disadvantage of lambda is that it decouples your economics from AWS's (or Azure's or GCP's) economics. That is, the cost of idle time is shifted from your account to their account. This sets up a bit of a race: Amazon's goal is to have as few hidden warm instances for as little time as possible. The user's goal is to have as many hidden warm instances as possible for as long as possible. This is where you start to see pinging to provoke AWS into keeping a warm copy. As time goes on they will have a growing incentive to prevent tricks like these from working. At the moment, on the economics, hosted FaaSes will always have this tension. Self-hosted FaaSes open up the possibility of setting your own policy for trading latency against idle cost. Disclosure: I am part of a team working on Project Riff, a FaaS.

Turbots 9y ago
I saw mark fishers riff demo at SpringOne and really liked it, just set up riff on my Mac with kubernetes on docker, I'm working on a fun demo right now, using kafka as a buffer is quite neat! Excited to see what this project will evolve into
merb 9y ago
> Self-hosted FaaSes open up the possibility of setting your own policy for trading latency against idle cost if you already have a K8s cluster it's probably cheaper to just create your application regulary and use it like that. No need to use a FaaS which complicates your whole setup for no additional benefit.
- jacques_chester 9y ago
  Self-hosted isn't free either. You're still paying for resources when they're idle. The difference is that you can decide on whether to scale down to 1 or not. As it happens Riff and most other FaaS projects are built on kubernetes.
  - merb 9y ago
    > Self-hosted isn't free either. You're still paying for resources when they're idle. I still have these resources idle, when I use Riff.