4 ms·

Awesome work, but I am genuinely curios about the use cases where the 200ms init time being a problem?

by moqizhengz 1y ago

ttoinou 1y ago
Maybe its a problem when you’re paying 200ms per small request and you have millions of them ?
- Eric_WVGG 1y ago
  Some companies who actually depend on actions like this should consider fronting some cash to keep this project going.
- bastawhiz 1y ago
  If you have cold starts on all of your millions of requests, something is very wrong with your code.
- hnlmorg 1y ago
  The hypothetical 200ms is for cold starts. That means when the docker container that runs behind the lambda first starts up. When a lambda is invocated, that docker container isn’t immediately destroyed. It’s kept around for reuse. So there isn’t a realistic scenario where you would get a million cold starts in a month.
ericmcer 1y ago
Serverless implementations where lambdas are being used for APIs so +200ms on a single call is not good.
- bastawhiz 1y ago
  It's not 200ms per call. It's 200ms cold start. Once a lambda is started, you don't pay that cost again. In most cases, you pay this a few times after a deployment or when traffic is increasing and not again.
  - hvb2 1y ago
    > you pay this a few times after a deployment or when traffic is increasing and not again. This is now how lambda works. A lambda instance sticks around for 10s of minutes at most. So you'll have cold starts every day no matter how often you deploy. If you have very little traffic you might actually have cold starts often as the lifetime of a lambda is determined based on its use. Unfortunately AWS has always been rather unspecific about this behavior but here's a link to a pluralsight blog https://www.pluralsight.com/resources/blog/cloud/how-long-does-aws-lambda-keep-your-idle-functions-around-before-a-cold-start https://www.pluralsight.com/resources/blog/cloud/how-long-do...
nemothekid 1y ago
I have a lambda that runs inference on a model for a feature that is seldomly used. Lambda is far more economic given the memory requirements, but we recently moved to Rust+Candle to shave off ~300ms on cold starts as the lag could be really jarring.

Awesome work, but I am genuinely curios about the use cases where the 200ms init time being a problem?

ttoinou 1y ago

Maybe its a problem when you’re paying 200ms per small request and you have millions of them ?

Eric_WVGG 1y ago
Some companies who actually depend on actions like this should consider fronting some cash to keep this project going.
bastawhiz 1y ago
If you have cold starts on all of your millions of requests, something is very wrong with your code.
hnlmorg 1y ago
The hypothetical 200ms is for cold starts. That means when the docker container that runs behind the lambda first starts up. When a lambda is invocated, that docker container isn’t immediately destroyed. It’s kept around for reuse. So there isn’t a realistic scenario where you would get a million cold starts in a month.

ericmcer 1y ago

Serverless implementations where lambdas are being used for APIs so +200ms on a single call is not good.

bastawhiz 1y ago
It's not 200ms per call. It's 200ms cold start. Once a lambda is started, you don't pay that cost again. In most cases, you pay this a few times after a deployment or when traffic is increasing and not again.
- hvb2 1y ago
  > you pay this a few times after a deployment or when traffic is increasing and not again. This is now how lambda works. A lambda instance sticks around for 10s of minutes at most. So you'll have cold starts every day no matter how often you deploy. If you have very little traffic you might actually have cold starts often as the lifetime of a lambda is determined based on its use. Unfortunately AWS has always been rather unspecific about this behavior but here's a link to a pluralsight blog https://www.pluralsight.com/resources/blog/cloud/how-long-does-aws-lambda-keep-your-idle-functions-around-before-a-cold-start https://www.pluralsight.com/resources/blog/cloud/how-long-do...

nemothekid 1y ago

I have a lambda that runs inference on a model for a feature that is seldomly used. Lambda is far more economic given the memory requirements, but we recently moved to Rust+Candle to shave off ~300ms on cold starts as the lag could be really jarring.