2 ms·
Sorry--the throttling was at the AWS Lambda layer, not S3. We were being throttled because we'd deploy a new container image and suddenly thousands of new conta
by moduspol 2y ago
Sorry--the throttling was at the AWS Lambda layer, not S3. We were being throttled because we'd deploy a new container image and suddenly thousands of new containers are all simultaneously trying to pull the database file from S3.
We aim to return a response in the single digit milliseconds and sometimes get tens of thousands of requests per second, so even if it only takes a second or two to fetch that file from S3, the request isn't getting served while it's happening, and new requests are coming in.
You very quickly hit your Lambda concurrency limit and get throttled just waiting for your instances to fetch the file, even though logically you're doing exactly what you planned to.
By having the file exist already in the container image, you lean on AWS's existing tools for a phased rollout to replace portions of your deployment at a time, and every one is responding in single digit milliseconds from its very first request.
EDIT: The same technique could be applied for other container management systems, but for stuff like Kubernetes or ECS, it might be simpler to use OP's method with a readiness check that only returns true if you fetched the file successfully. And maybe some other logic to do something if your file gets too stale, or you're failing to fetch updates for some reason.
- ec109685 2y agoAre lambdas guaranteed to be warm on the first request to serve in single digit milliseconds?
- rajamaka 2y agoThey can be with provisioned concurrency.
- ec109685 2y agoIf provisioned concurrency is used, then fetching the database at startup would be a valid strategy given you could still achieve flat response times.
- moduspol 2y agoThat's true. I prefer this approach because it removes that additional thing (the request to S3) that can be slow or fail at runtime. Or "initialization" time, I guess, depending on how you look at it.