3 ms·
We're using Nomad, and appreciate its comparative simplicity (we launch plain binaries created with Nix), but there are still basic features missing that can cr
by nh2 3y ago
We're using Nomad, and appreciate its comparative simplicity (we launch plain binaries created with Nix), but there are still basic features missing that can create struggles. The ones I hit are:
* It can't work down batch jobs in priority order [1] -- a feature even the ultra-old university cluster schedulers had
* Sometimes job starts can be in arbitrary order, in the API and the UI, making it difficult to find the most recent ones [2]
* Nomad seems to store pretty much everything in its Raft state, making it extremely slow to startup and operate (including GUI) if some rogue process created 100k allocations. Cleaning that up seems to require nuking the Raft state, thus also losing all info about all jobs.
I don't know how well Kubernetes does in these areas.
[1]: https://github.com/hashicorp/nomad/issues/12792 https://github.com/hashicorp/nomad/issues/12792
[2]: https://github.com/hashicorp/nomad/issues/17742 https://github.com/hashicorp/nomad/issues/17742
- k8svet 3y agoSomeone I know once worked for the company maintaining etcd at the time and... they were appalled at how it was underpinning millions of dollars of VC dreams on it, implicitly via kubernetes, with the flaws and downsides, and operational complexities it has. And now to hear you say this of Nomad and Raft. I guess, my take away at this second is to really be skeptical of hype that I don't have years of operational knowledge and experience with. Which, to be fair, regarding my current tech du jour (well, 9+ years), I have a list of complaints that could span a short novela.