3 ms·
DynamoDB happily handled 20 million writes per hour, but our error rate on Lambda was spiking as soon as we went above 1 million/hour invocations, and we were n
by yarapavan 9y ago
DynamoDB happily handled 20 million writes per hour, but our error rate on Lambda was spiking as soon as we went above 1 million/hour invocations, and we were not being throttled. Looking at the logs, we quickly understood what was happening. We were overwhelming the S3 bucket with PUT requests, and we were receiving this response back: "Please reduce your request rate."
We opened a ticket with AWS and quickly discovered that S3 will limit your write throughput to a bucket, especially if the keys that you are writing happen to be in alphabetic, sequential order — which was exactly what we were doing.