5 ms·
DynamoDB streams are pretty awesome and my company uses it for all kinds of neat aggregation scenarios. The one major thing to consider with streams is data los
by sakopov 5y ago
DynamoDB streams are pretty awesome and my company uses it for all kinds of neat aggregation scenarios. The one major thing to consider with streams is data loss and how it can impact your use case. Any non-intermittent issues in the lambda will cause stream processing to stall out and I believe DynamoDB will begin shedding events after 24 hours by default, so you will experience data loss unless you take certain actions to prevent it. I would highly recommend using IteratorAge metric in your lambda to track times when your lambda is falling behind in processing of stream events. From my personal experience, it is sometimes a better option to write stream events to an SQS queue instead of processing directly from the stream, as it offers better durability and better dead-lettering/recovery support, and then do aggregation tasks on the queue. Just my 0.02.