4 ms·

Thanks for the reply! Not really an answer to the scheduler question, but at least it mirrors some of my experience. That K8s is something to avoid, and that

Thanks for the reply! Not really an answer to the scheduler question, but at least it mirrors some of my experience. That K8s is something to avoid, and that it does not scale, is a known (at least to me). But that doesn't answer what people would put on the metal when building DCs… I was not asking out of the perspective of an end-user. I was asking about (large) DC scale infra. (As dev I know the end-user stuff). As I see it: You can build your own stuff from scratch (which is not realistic in most cases I guess), or you can use OpenStack or Mesos. There are no more alternatives at the moment I think, and it's unlikely that someone comes up with something new. OTOHS that's OK. A lot of people will never need to build their own DC(s). For smaller setups (say one or two racks) there are more options of course. (You could run for example Proxmox and maybe something on top).

KaiserPro 4y ago
Sorry yeah, I didn't really answer your question. Here is a non-exhaustive list of schedulers for differing use cases: https://slurm.schedmd.com/documentation.html https://slurm.schedmd.com/documentation.html << mostly batch, not entirely convinced it actually scales that well https://www.altair.com/grid-engine https://www.altair.com/grid-engine << originally it was sun's grid engine. There are a number of offshoots tuned for different scenarios. https://rmanwiki.pixar.com/display/TRA/Tractor+2 https://rmanwiki.pixar.com/display/TRA/Tractor+2 << thats for the VFX crowd https://www.opencue.io/ https://www.opencue.io/ << open source which is vaguely related to the above https://abelay.github.io/6828seminar/papers/hazelwood:ml.pdf https://abelay.github.io/6828seminar/papers/hazelwood:ml.pdf << Facebook's version of airflow. It sits on top of two schedulers, so its not really a good fit. I can;t find what they publicly call their version of the borg I'm assuming you've read about borg As you've pointed out mesos is there as well.
- still_grokking 4y ago
  Cool! Thanks! That's a lot of stuff I didn't hear about until now. It's really nice that one can meet experts here on HN and get free valuable answers form them. Thank you.

KaiserPro 4y ago

Sorry yeah, I didn't really answer your question. Here is a non-exhaustive list of schedulers for differing use cases: https://slurm.schedmd.com/documentation.html https://slurm.schedmd.com/documentation.html << mostly batch, not entirely convinced it actually scales that well https://www.altair.com/grid-engine https://www.altair.com/grid-engine << originally it was sun's grid engine. There are a number of offshoots tuned for different scenarios. https://rmanwiki.pixar.com/display/TRA/Tractor+2 https://rmanwiki.pixar.com/display/TRA/Tractor+2 << thats for the VFX crowd https://www.opencue.io/ https://www.opencue.io/ << open source which is vaguely related to the above https://abelay.github.io/6828seminar/papers/hazelwood:ml.pdf https://abelay.github.io/6828seminar/papers/hazelwood:ml.pdf << Facebook's version of airflow. It sits on top of two schedulers, so its not really a good fit. I can;t find what they publicly call their version of the borg I'm assuming you've read about borg As you've pointed out mesos is there as well.

still_grokking 4y ago
Cool! Thanks! That's a lot of stuff I didn't hear about until now. It's really nice that one can meet experts here on HN and get free valuable answers form them. Thank you.