38 ms·
Why in the world do you need a Kubernetes deployment system to run a single, manual, one-time (or a handful of times), high-compute job?
by _bxg1 7y ago
Why in the world do you need a Kubernetes deployment system to run a single, manual, one-time (or a handful of times), high-compute job?
- PeterisP 7y agoBecause that high-compute job needs to be distributed on many, many machines, and if you're using cheap preemptible instances you have to handle machines dropping off and joining in while you're running that single job. It's definitely not something that you can launch manually - perhaps Kubernetes is not the best solution, but you definitely need some automation.
- dsl 7y agoBecause when all you have is a hammer, everything looks like a nail. We have become so DevOps and cloud dependent that everyone has forgotten how to just run big systems cheaply and efficiently.
- KaiserPro 7y agobecause its not a one time operation. also, how else do you sensibly deploy and manage a multi-stage programme on >500 nodes? I mean we use AWS batch, which is far superior for this sort of thing. SLURM might work for real steel, as would tractor from pixar.