5 ms·

Listen folks - it's still a fantasy. Latency is still 5s on many requests = no deal. This has been going on for years. Would someone form AMZ please stand up?

Listen folks - it's still a fantasy. Latency is still 5s on many requests = no deal. This has been going on for years. Would someone form AMZ please stand up? I suggest that the latency is 5s for the 'first call', then, subsequent calls are fast - and if you don't use your lambda for a minute or so, it goes back to having latency. I suggest maybe this has something to do with loading it into memory when it's used? There must be a way to keep the lambdas 'hot' or else it will never really get used for anything but background tasks.

toomuchtodo 10y ago
You break the entire model of an oversubscribed multi-tenant container system if you want every lambda hot all the time. If that's what you need...you need a server.
- jomamaxx 10y ago
  "If that's what you need...you need a server." We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again. "You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.
  - deleted 10y ago
    [deleted]
  - toomuchtodo 10y ago
    > We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again. I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible. Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.
    - jomamaxx 10y ago
      What I'm asking for is definitely within grasp of Amazon's team :) they just need to get there. Moreover, it's a rather fundamental opportunity for them: Lambda's are a form of 'container' and they are exactly what so many of us want. The deal-breaker for us was the latency issue. Amazon is a pretty smart company, and it's a big opportunity, I'll bet they eventually get this sorted out.
  - Sanddancer 10y ago
    What you're describing is managed hosting. In order to do it, you either have to oversubscribe like hell, and deal with latency, or pay for wide open swaths of server. Pick your poison.
- imtringued 10y ago
  Keeping a Lambda container in memory is probably not going to take more than 500MB of RAM. Paying a monthly minimum fee per lambda per machine per month to keep it in RAM should be an option. It would still be far cheaper than a real server.
  - toomuchtodo 10y ago
    Would you be able to explain why it'd be far cheaper than a real virtual machine? Because the ram is the most expensive part in both cases.
ranman 10y ago
stands up https://www.youtube.com/watch?v=eJO5HU_7_1w https://www.youtube.com/watch?v=eJO5HU_7_1w You're describing container re-use which is documented in the page here: http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction-function.html http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct... You could have cloud watch run the function every minute? Again, that won't guarantee that your function will be "hot". But it's a quick hack to potentially reduce the frequency of your slow initial start. That said customer feedback is super helpful in making great services so I'll get this feedback to the lambda team (or they may see it here). If you ever need any help my email is in my profile and I'm happy to work with you.
- jomamaxx 10y ago
  Thanks for your response. I checked out the doc and didn't see any reference to latency or how to mitigate it, perhaps I'm missing something? Listen - I'm a massive fan of AWS. We use some core services and are very happy. I know you listen to customers - but I suggest that Lambda was probably designed for specific kind of services (maybe IT, admin, batch processing etc.) - I'm going to fathom that maybe 'people running web-sites' were top of mind? Again, purely speculation on my part. But if you want people to seriously switch over - and they will - in droves - if you get it right, is to basically enable fairly responsive services that scale with a minimum of fuss, latency concerns included. So - if I'm running a web-portal - and I have a customer pull some data to build a chart from somewhere - and I have a Lambda specifically geared for that microservice - then it needs to be quick. There really should be no need to use other services to have to hack around the latency issue. Surely - you can parameterize the feature if you want, and require users to set thresholds for how long Lambda's 'stay hot' (and forgive my crude metaphors), or how much throughput etc. - and you could even bill them based upon those parameters. As long as it's fairly concise and understandable. Perhaps DynamoDB is a good example. We've used it before - and what we love about it is that as long as we design our data correctly we never have to worry. Bursts of traffic, etc. are generally handled well, it scales largely without concern. One poor limitation of that service is the requirement to indicate throughput thresholds, which is too bad, but beyond that, it's ideal. Amazon Lambdas are basically the 'last piece' of the golden dream of cloud computing - the ability to develop a truly serverless, microserver-based infrastructure, and never have to worry about platforms again. As much as we love EC2s, we would drop them instantly and go to Lambdas if they worked well. (And it's been at least 18 months since we last experimented, BTW). There are tons of SMEs that would do the same - I don't think anyone who isn't directly in the business of tech, and is providing some other kind of consumer or business service, wants the pain of managing EC2s or servers. 'Elastic Beanstalk' etc. is a bandaid - it's not ideal. Anyhow - Lambdas are very close to being a super great thing, more than that - transformational - if they could just cross that 'real world' usability threshold. Again - if something has changed WRT latency, and there is something in those docs I'm missing, please let me know.
jotto 10y ago
I measured the actual latency over the weekend [0] [1] Bare ec2: 15ms Lambda via API Gateway: 276ms [0]: http://prerender.cloud/lambda-latency http://prerender.cloud/lambda-latency [1]: https://news.ycombinator.com/item?id=12678011 https://news.ycombinator.com/item?id=12678011
- jomamaxx 10y ago
  Thanks for the data points. I understand that Lambdas are 'relatively quick' when they are 'hot' - which is my crude metaphor for 'up and running'. The issue is, if the Lambda has not been in use for some time (seconds, minutes?), then the 'first call' can have quite a bit of latency - often up to 5 seconds - which killed it for us. There is no way we could reasonably deploy a service wherein customer would have to wait an additional 5 seconds for a reaction in most use cases. So close, yet so far. This was about 18 months ago. If you have some data that addresses this specific artifact - then that would be great. Possibly things have changed in 18 months. And I'm not a hater, to us AWS is the best thing since sliced bread.
  - jotto 10y ago
    Actually I was making the point that they are not "relatively quick" even when hot: close to 300ms, almost entirely due to the API Gateway. (I even measured API Gateway on top of bare ec2 and the same penalty applies). Lambda itself is fairly quick (~75ms), but API Gateway needs some work.
    - jomamaxx 10y ago
      Yeah, I tend to agree, I just didn't want to be a total jerkoff on the thread. 250ms we can live with for many things, but not 5x. But yes - it should be under 50ms at all times I think for people to start taking it seriously.
  - brianwawok 10y ago
    The 5 second cold request thing is what GAE had 5 years ago. Eventually they gave up trying to make startup faster and allowed a paid plan to stay always hot. Which was good, but then also moving you much closer to a managed server (vs a pure pay as you use thing)
jasoncchild 10y ago
If you are serving high traffic then what's the problem? If your product has only occasional use then you trade cost for latency; spin up an ec2 instance if your request times exceed a few minutes call
- dx034 10y ago
  You could have certain requests that don't come that often, even on a high traffic site. And if you don't know for how long lambdas stay hot, how are you supposed to time it? Even a call every few minutes may not be enough to keep them hot.
  - jasoncchild 10y ago
    Ah, I see. Perhaps that would be a reason to keep an ec2 instance spun up to host the functionality that must always have latency < warmup+runtime? Surely suboptimal. This makes me consider running a cron job to poke timing critical lambda workflows every few minutes...which to me imply writing my functions in such away as to expect this "heartbeat" and act accordingly...and that just feels plain icky...

toomuchtodo 10y ago

You break the entire model of an oversubscribed multi-tenant container system if you want every lambda hot all the time. If that's what you need...you need a server.

jomamaxx 10y ago
"If that's what you need...you need a server." We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again. "You break the entire model of an oversubscribed multi-tenant container system" - we don't care that much about how it's implemented, or what underlying model is used, as long as it is secure, robust, and reliable.
- deleted 10y ago
  [deleted]
- toomuchtodo 10y ago
  > We want micro-services that are available as we need them, at any scale, and we never want to deal with a 'server' again. I'm going to argue that's currently unrealistic, but if you want someone to build something, tell them its impossible. Clearly its not easy to build a system that'll run code on demand as fast as bare metal; accept the trade offs with the understanding why you're making them.
  - jomamaxx 10y ago
    What I'm asking for is definitely within grasp of Amazon's team :) they just need to get there. Moreover, it's a rather fundamental opportunity for them: Lambda's are a form of 'container' and they are exactly what so many of us want. The deal-breaker for us was the latency issue. Amazon is a pretty smart company, and it's a big opportunity, I'll bet they eventually get this sorted out.
- Sanddancer 10y ago
  What you're describing is managed hosting. In order to do it, you either have to oversubscribe like hell, and deal with latency, or pay for wide open swaths of server. Pick your poison.
imtringued 10y ago
Keeping a Lambda container in memory is probably not going to take more than 500MB of RAM. Paying a monthly minimum fee per lambda per machine per month to keep it in RAM should be an option. It would still be far cheaper than a real server.
- toomuchtodo 10y ago
  Would you be able to explain why it'd be far cheaper than a real virtual machine? Because the ram is the most expensive part in both cases.

ranman 10y ago

stands up https://www.youtube.com/watch?v=eJO5HU_7_1w https://www.youtube.com/watch?v=eJO5HU_7_1w You're describing container re-use which is documented in the page here: http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction-function.html http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduct... You could have cloud watch run the function every minute? Again, that won't guarantee that your function will be "hot". But it's a quick hack to potentially reduce the frequency of your slow initial start. That said customer feedback is super helpful in making great services so I'll get this feedback to the lambda team (or they may see it here). If you ever need any help my email is in my profile and I'm happy to work with you.

jomamaxx 10y ago
Thanks for your response. I checked out the doc and didn't see any reference to latency or how to mitigate it, perhaps I'm missing something? Listen - I'm a massive fan of AWS. We use some core services and are very happy. I know you listen to customers - but I suggest that Lambda was probably designed for specific kind of services (maybe IT, admin, batch processing etc.) - I'm going to fathom that maybe 'people running web-sites' were top of mind? Again, purely speculation on my part. But if you want people to seriously switch over - and they will - in droves - if you get it right, is to basically enable fairly responsive services that scale with a minimum of fuss, latency concerns included. So - if I'm running a web-portal - and I have a customer pull some data to build a chart from somewhere - and I have a Lambda specifically geared for that microservice - then it needs to be quick. There really should be no need to use other services to have to hack around the latency issue. Surely - you can parameterize the feature if you want, and require users to set thresholds for how long Lambda's 'stay hot' (and forgive my crude metaphors), or how much throughput etc. - and you could even bill them based upon those parameters. As long as it's fairly concise and understandable. Perhaps DynamoDB is a good example. We've used it before - and what we love about it is that as long as we design our data correctly we never have to worry. Bursts of traffic, etc. are generally handled well, it scales largely without concern. One poor limitation of that service is the requirement to indicate throughput thresholds, which is too bad, but beyond that, it's ideal. Amazon Lambdas are basically the 'last piece' of the golden dream of cloud computing - the ability to develop a truly serverless, microserver-based infrastructure, and never have to worry about platforms again. As much as we love EC2s, we would drop them instantly and go to Lambdas if they worked well. (And it's been at least 18 months since we last experimented, BTW). There are tons of SMEs that would do the same - I don't think anyone who isn't directly in the business of tech, and is providing some other kind of consumer or business service, wants the pain of managing EC2s or servers. 'Elastic Beanstalk' etc. is a bandaid - it's not ideal. Anyhow - Lambdas are very close to being a super great thing, more than that - transformational - if they could just cross that 'real world' usability threshold. Again - if something has changed WRT latency, and there is something in those docs I'm missing, please let me know.

jotto 10y ago

I measured the actual latency over the weekend [0] [1] Bare ec2: 15ms Lambda via API Gateway: 276ms [0]: http://prerender.cloud/lambda-latency http://prerender.cloud/lambda-latency [1]: https://news.ycombinator.com/item?id=12678011 https://news.ycombinator.com/item?id=12678011

jomamaxx 10y ago
Thanks for the data points. I understand that Lambdas are 'relatively quick' when they are 'hot' - which is my crude metaphor for 'up and running'. The issue is, if the Lambda has not been in use for some time (seconds, minutes?), then the 'first call' can have quite a bit of latency - often up to 5 seconds - which killed it for us. There is no way we could reasonably deploy a service wherein customer would have to wait an additional 5 seconds for a reaction in most use cases. So close, yet so far. This was about 18 months ago. If you have some data that addresses this specific artifact - then that would be great. Possibly things have changed in 18 months. And I'm not a hater, to us AWS is the best thing since sliced bread.
- jotto 10y ago
  Actually I was making the point that they are not "relatively quick" even when hot: close to 300ms, almost entirely due to the API Gateway. (I even measured API Gateway on top of bare ec2 and the same penalty applies). Lambda itself is fairly quick (~75ms), but API Gateway needs some work.
  - jomamaxx 10y ago
    Yeah, I tend to agree, I just didn't want to be a total jerkoff on the thread. 250ms we can live with for many things, but not 5x. But yes - it should be under 50ms at all times I think for people to start taking it seriously.
- brianwawok 10y ago
  The 5 second cold request thing is what GAE had 5 years ago. Eventually they gave up trying to make startup faster and allowed a paid plan to stay always hot. Which was good, but then also moving you much closer to a managed server (vs a pure pay as you use thing)

jasoncchild 10y ago

If you are serving high traffic then what's the problem? If your product has only occasional use then you trade cost for latency; spin up an ec2 instance if your request times exceed a few minutes call

dx034 10y ago
You could have certain requests that don't come that often, even on a high traffic site. And if you don't know for how long lambdas stay hot, how are you supposed to time it? Even a call every few minutes may not be enough to keep them hot.
- jasoncchild 10y ago
  Ah, I see. Perhaps that would be a reason to keep an ec2 instance spun up to host the functionality that must always have latency < warmup+runtime? Surely suboptimal. This makes me consider running a cron job to poke timing critical lambda workflows every few minutes...which to me imply writing my functions in such away as to expect this "heartbeat" and act accordingly...and that just feels plain icky...