4 ms·
Another option is to over provisioned spot instances, such that it would feel like a 24/7 instance, but cost like spot. However, you would probably need an auto
by streetcat1 7y ago
Another option is to over provisioned spot instances, such that it would feel like a 24/7 instance, but cost like spot. However, you would probably need an automatic platform to achieve that.
- minimaxir 7y agoAn idea I've had is to use GCP preemptibles + script to automatically run a ML training job on instance start + Google Cloud Function + Cloud Scheduler to attempt to start the instance every minute if it gets preempted. The latter two are effectively free, so you'd get the cost benefits of preemptibles as long as the ML training job is resilient to random shutdowns.
- streetcat1 7y agoYes, this is basically the idea. However, there are different solutions for training and inference. For training, I would recommend that you add automatic checkpoint, and even consider model migration. For inference (which I think is the original concern), over provisioning is the key (simply because the fact that it would take a long time to load the model. Also, you also want to diversify your node types, etc.