4 ms·
If you want sequential processing of the data in the SQS queue, something which works really well today is to create a state machine in AWS Step Functions which
by Dunedan 6y ago
If you want sequential processing of the data in the SQS queue, something which works really well today is to create a state machine in AWS Step Functions which triggers a AWS Lambda function which then pulls data from SQS and processes it. Using a condition in the state machine, this can be done in a loop, so when the AWS Lambda function reaches its timeout, another one gets triggered as long as there is still data in the SQS queue.
If data doesn't have to be processed sequentially an option is to configure the AWS Lambda function to get invoked for new data in the SQS queue [1], so you don't have to care about manually fetching data from SQS at all.
[1]: https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html
- staticassertion 6y agoYep, but I prefer to care about manually fetching data from SQS. It's a weird system, but due to our data model there are many benefits to processing as many messages in a given lambda as possible.
- kondro 6y agoThis might be useful to simplify that model for you :) https://aws.amazon.com/about-aws/whats-new/2020/11/aws-lambda-now-supports-batch-windows-of-up-to-5-minutes-for-functions/ https://aws.amazon.com/about-aws/whats-new/2020/11/aws-lambd...
- staticassertion 6y agoPretty cool, but not quite the model I want.