3 ms·

I've solved the replaying bit before with a brute approach and AWS Athena. It would ingest all the events from S3, filter the unwanted ones out, and put the res

I've solved the replaying bit before with a brute approach and AWS Athena. It would ingest all the events from S3, filter the unwanted ones out, and put the rest in SQS ready for consumption. It was definitely expensive though, not something you would run often.

uzarubin 6y ago
This is definitely a valid approach and it is an expensive path. Athena charges a hefty amount per query if you have a large dataset. In addition, this approach won't work if your data is serialized with something like protobuf.