You can get rid of Step 2 if you're going to be creating a new S3 object for each batch. AWS has an inbuilt feature to trigger SQS notifications for S3 operatio

You can get rid of Step 2 if you're going to be creating a new S3 object for each batch. AWS has an inbuilt feature to trigger SQS notifications for S3 operations: https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-event-notifications.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable...