5 ms·

My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are desi

My experience with cold starts in Azure Functions Serverless is pretty awful. Like most other Azure services, their affordable consumer grade offerings are designed from the ground up not to be good enough for "serious" use. Cold start times compared to Lambda are worse, and in addition, we would get random 404s which do not appear in any logs; inspecting these 404s indicated they were emitted by nginx, leading me to believe that the ultimate container endpoint was killed for whatever reason but that fact didn't make it back to the router, which attempted and failed to reach the function. Of course the cold start and 404 are mitigated if you pay for the premium serverless or just host their middleware on their own App Service plans (basically VMs)

secondaryacct 5y ago
My company is moving away from datacenter and into Azure and I have to get the az900 this week, it doesnt bode well. And I was so happy to leave the clusterfuck of 300 aws lambda I was working with in my prev company. What an expensive fad, and no engineer is ever consulted ...
- cmcconomy 5y ago
  There are things I like, such as the consistent and pervasive security model with Azure AD. I don't even mind running stuff on App Services. I just find their commodity serverless offering in particular is subpar
  - meekins 5y ago
    > consistent and pervasive security model with Azure AD Wait, this is the first time I hear this about Azure. Could you elaborate? It is possible that things have improved significantly since I last worked with Azure but lack of a consistent security model (like IAM on AWS) to control human and service (Azure Functions, App Service apps etc) access to specific resources (Cosmos databases, EventHubs etc) especially painful.
    - cmcconomy 5y ago
      some of it is wonky, such as the login model for postgres on Azure SQL (you create login-capable postgres groups that exactly mirror the name of an Azure AD group, and then the "password" you pass in is actually a JWT proving YOU are in fact a member of that AD group -- so you have to hit a funky endpoint to get a time-limited "password")
- dharmab 5y ago
  Never use Basic SKU, plan your network carefully before you even create the vnets, monitor NAT capacity, beware undocumented ARM rate limits. Good luck
- jamesfinlayson 5y ago
  > 300 aws lambda I'm living that now but moving it out of Lambda and into ECS it as fast as I can.
adflux 5y ago
The way Azure Function scales out is different and is not entirely suited for the same goal as lambdas. Lambdas happily scale from 1 to 1000 instances in seconds* (EDIT: not A second), whereas Azure Functions just wont do that.
- VWWHFSfQ 5y ago
  > Lambdas happily scale from 1 to 1000 instances in a second What on earth. No they don't...
  - adflux 5y ago
    https://docs.aws.amazon.com/lambda/latest/dg/invocation-scaling.html https://docs.aws.amazon.com/lambda/latest/dg/invocation-scal... Last time I tried this was a few years ago, but seems like its still the case. For an initial burst of traffic, your functions' cumulative concurrency in a Region can reach an initial level of between 500 and 3000, which varies per Region. After the initial burst, your functions' concurrency can scale by an additional 500 instances each minute. This continues until there are enough instances to serve all requests, or until a concurrency limit is reached. When requests come in faster than your function can scale, or when your function is at maximum concurrency, additional requests fail with a throttling error (429 status code).
    - VWWHFSfQ 5y ago
      That's just burst concurrency. It takes a lot longer that 1 second to actually reach the peak burstable limit. So while the Lambdas are scaling up your clients are either waiting several seconds, or getting 429s.
danielvaughn 5y ago
Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea. The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking cold start.
- kaishiro 5y ago
  I'm assuming by Firebase you mean Firebase Functions? We have a fairly complex infrastructure running off a combination of Firebase's RTDB, Firestore, and Google Cloud Functions and have never seen anywhere near what you're describing. Are you sure you're experiencing a 7 second "cold start", or is the invocation simply taking 7 seconds to run? Because the latter is far more easily explained.
  - tenaciousDaniel 5y ago
    I've confirmed that the actual execution of the function itself takes ~30ms, and that the time to download is standard (~200ms). That only leaves the cold start; nothing else makes sense. edit: I even set up a node server on AppEngine itself, copied over the exact code to an endpoint, and it was taking 300-500ms tops (after the initial cold start of the AppEngine server).
- earthboundkid 5y ago
  7s cold start is weird. I think there has to be more to the story than that.
  - alex_duf 5y ago
    I've seen 30s on AWS, so it's not that surprising. They have now improved it greatly though. And yet I still believe it's a great technology, as always it's a matter of putting it on the right use case. Message consumption from a queue or topic, low traffic and low criticality API are two great use cases.
HappyVertical 5y ago
For the Azure Functions consumption plan this can be mitigated to an extent by just having a keep alive function run inside the same function app (set to say a 3-5 minute timer trigger).
GordonS 5y ago
I like Azure in general, but Function cold start times are really awful. I regularly see start up times exceeding 10s for small, dotnet based functions. One is an auth endpoint for a self-hosted Docker registry, and the Docker CLI often times out when logging in if there is a cold start. I'm planning on moving these functions to Docker containers hosted in a VM. I have other issues with Functions too. If you enable client certificates, the portal UI becomes pretty useless, with lots of stuff inaccessible. I have one such endpoint in production just now, and it's even worse than that, as every now and then it just... stops working until I manually restart it. Nothing useful in the logs either.
AtNightWeCode 5y ago
Azure Functions, in my opinion, should mostly be used in cases when you want to do some work over some time every now and then. It will also probably be cheaper to use something else in your case. In later versions of AF you can use a real Startup file to mitigate some life cycle related issues.
MaKey 5y ago
Azure Functions cold start times also depend on the underlying tech stack. I was using Python on a Linux host for Slack related Azure Functions and they ran into timeouts sometimes (which for the Slack API is 3s I think). After I switched to Nodejs on Windows I never got a timeout again.

secondaryacct 5y ago

My company is moving away from datacenter and into Azure and I have to get the az900 this week, it doesnt bode well. And I was so happy to leave the clusterfuck of 300 aws lambda I was working with in my prev company. What an expensive fad, and no engineer is ever consulted ...

cmcconomy 5y ago
There are things I like, such as the consistent and pervasive security model with Azure AD. I don't even mind running stuff on App Services. I just find their commodity serverless offering in particular is subpar
- meekins 5y ago
  > consistent and pervasive security model with Azure AD Wait, this is the first time I hear this about Azure. Could you elaborate? It is possible that things have improved significantly since I last worked with Azure but lack of a consistent security model (like IAM on AWS) to control human and service (Azure Functions, App Service apps etc) access to specific resources (Cosmos databases, EventHubs etc) especially painful.
  - cmcconomy 5y ago
    some of it is wonky, such as the login model for postgres on Azure SQL (you create login-capable postgres groups that exactly mirror the name of an Azure AD group, and then the "password" you pass in is actually a JWT proving YOU are in fact a member of that AD group -- so you have to hit a funky endpoint to get a time-limited "password")
dharmab 5y ago
Never use Basic SKU, plan your network carefully before you even create the vnets, monitor NAT capacity, beware undocumented ARM rate limits. Good luck
jamesfinlayson 5y ago
> 300 aws lambda I'm living that now but moving it out of Lambda and into ECS it as fast as I can.

adflux 5y ago

The way Azure Function scales out is different and is not entirely suited for the same goal as lambdas. Lambdas happily scale from 1 to 1000 instances in seconds* (EDIT: not A second), whereas Azure Functions just wont do that.

VWWHFSfQ 5y ago
> Lambdas happily scale from 1 to 1000 instances in a second What on earth. No they don't...
- adflux 5y ago
  https://docs.aws.amazon.com/lambda/latest/dg/invocation-scaling.html https://docs.aws.amazon.com/lambda/latest/dg/invocation-scal... Last time I tried this was a few years ago, but seems like its still the case. For an initial burst of traffic, your functions' cumulative concurrency in a Region can reach an initial level of between 500 and 3000, which varies per Region. After the initial burst, your functions' concurrency can scale by an additional 500 instances each minute. This continues until there are enough instances to serve all requests, or until a concurrency limit is reached. When requests come in faster than your function can scale, or when your function is at maximum concurrency, additional requests fail with a throttling error (429 status code).
  - VWWHFSfQ 5y ago
    That's just burst concurrency. It takes a lot longer that 1 second to actually reach the peak burstable limit. So while the Lambdas are scaling up your clients are either waiting several seconds, or getting 429s.

danielvaughn 5y ago

Same experience with Firebase. I just joined a team that has been using it. I've never worked with serverless before, and it boggles my mind how anyone thought it would be a good idea. The cold starts are horrendous. In one case, it's consistently taking about 7 seconds to return ~10K of data. I investigated the actual runtime of the function and it completes in about 20ms, so the only real bottleneck is the fucking cold start.

kaishiro 5y ago
I'm assuming by Firebase you mean Firebase Functions? We have a fairly complex infrastructure running off a combination of Firebase's RTDB, Firestore, and Google Cloud Functions and have never seen anywhere near what you're describing. Are you sure you're experiencing a 7 second "cold start", or is the invocation simply taking 7 seconds to run? Because the latter is far more easily explained.
- tenaciousDaniel 5y ago
  I've confirmed that the actual execution of the function itself takes ~30ms, and that the time to download is standard (~200ms). That only leaves the cold start; nothing else makes sense. edit: I even set up a node server on AppEngine itself, copied over the exact code to an endpoint, and it was taking 300-500ms tops (after the initial cold start of the AppEngine server).
earthboundkid 5y ago
7s cold start is weird. I think there has to be more to the story than that.
- alex_duf 5y ago
  I've seen 30s on AWS, so it's not that surprising. They have now improved it greatly though. And yet I still believe it's a great technology, as always it's a matter of putting it on the right use case. Message consumption from a queue or topic, low traffic and low criticality API are two great use cases.

HappyVertical 5y ago

For the Azure Functions consumption plan this can be mitigated to an extent by just having a keep alive function run inside the same function app (set to say a 3-5 minute timer trigger).

GordonS 5y ago

I like Azure in general, but Function cold start times are really awful. I regularly see start up times exceeding 10s for small, dotnet based functions. One is an auth endpoint for a self-hosted Docker registry, and the Docker CLI often times out when logging in if there is a cold start. I'm planning on moving these functions to Docker containers hosted in a VM. I have other issues with Functions too. If you enable client certificates, the portal UI becomes pretty useless, with lots of stuff inaccessible. I have one such endpoint in production just now, and it's even worse than that, as every now and then it just... stops working until I manually restart it. Nothing useful in the logs either.

AtNightWeCode 5y ago

Azure Functions, in my opinion, should mostly be used in cases when you want to do some work over some time every now and then. It will also probably be cheaper to use something else in your case. In later versions of AF you can use a real Startup file to mitigate some life cycle related issues.

MaKey 5y ago

Azure Functions cold start times also depend on the underlying tech stack. I was using Python on a Linux host for Slack related Azure Functions and they ran into timeouts sometimes (which for the Slack API is 3s I think). After I switched to Nodejs on Windows I never got a timeout again.