Have you considered a cron lambda that breaks the task down into smaller parts and creates jobs for those parts onto SQS or SNS? Sounds like you already have sm

Have you considered a cron lambda that breaks the task down into smaller parts and creates jobs for those parts onto SQS or SNS? Sounds like you already have smaller jobs since you are breaking up to threads. You could then have your worker lambdas being triggered off the SNS publication and doing the work.