First time I hear about Elasticluster. Regarding cluster provisioning, auto-scaling is critical for cost and convenience. We managed to do this with a simple jo

First time I hear about Elasticluster. Regarding cluster provisioning, auto-scaling is critical for cost and convenience. We managed to do this with a simple job queue that outputs a CloudWatch metric that in turn controls an auto-scaling group of spot instances. Works like a charm but AWS Batch should make this redundant. We took a look at Airflow, which looks nice, but we haven't had a compelling reason not to use Luigi.