If you use Databricks, it's pretty easy to schedule a Databricks notebook to run. AWS EMR also has notebook support, I haven't used it, but it might be possible

If you use Databricks, it's pretty easy to schedule a Databricks notebook to run. AWS EMR also has notebook support, I haven't used it, but it might be possible to schedule those too. You would still have to grant permission to the Databricks role to write to a production location in S3. I really dislike this practice. Code that runs in production should be code reviewed and there should be some monitoring in place to make sure the job is working correctly