No, we have not used auto-scaling with automatic hardware provisioning. We assume that the hardware is there and we just request more containers (e.g, YARN cont

No, we have not used auto-scaling with automatic hardware provisioning. We assume that the hardware is there and we just request more containers (e.g, YARN containers). Theoretically, it should be possible to do that if the autoscaling policy is extended to request for additional hardware before changing the parallelism of a stage. But this functionality is not currently there.