3 ms·
This is essentially what I have in mind - some kind of queue, with a mediator that starts/stops a spot instance using a custom image that has everything install
by mnahkies 4y ago
This is essentially what I have in mind - some kind of queue, with a mediator that starts/stops a spot instance using a custom image that has everything installed and configured to start on boot
With regards to state I just meant the image - wouldn't want to have to install everything at each start. Using something like aws or gcp I don't think this would be an issue, but I was window shopping some providers that specialize in GPU VMs and it wasn't clear if they provided equivalent services. Probably easiest to just use aws.
- stavros 4y agoI think if you couldn't make a ready-made image you'd use a Docker container with everything in it instead. Though then I don't know how you'd use spot instances, because AFAIK you want those for the "whenever it's available" startup and can't really be logging into each one to start them up. If they let you run a container, presumably they let you run other commands to set stuff up as well? This kind of depends on the provider, but it's mostly an implementation detail. The main crux is a broker and a queue, I use Dramatiq (the site is written in Python) and it works great.
- mnahkies 4y agoWith spot instances you just have to be prepared for them to get terminated at random. If you're processing messages off a queue in most cases you get this for free - the ack deadline will expire and the message will be redelivered to another subscriber. Docker container might be a good shout, though I've not done anything using a GPU from docker before
- stavros 4y agoYeah exactly, the queue handles this for free. Docker with the Nvidia extension actually works better for me than native, Torch doesn't work on my system but works in Docker.