4 ms·

We're all-in on kubernetes and cri where I work, but Nomad is cool and if we were starting out now we'd certainly take a close look at it. We use other Hashicor

We're all-in on kubernetes and cri where I work, but Nomad is cool and if we were starting out now we'd certainly take a close look at it. We use other Hashicorp tools, terraform for example, so we're already familiar with HCL (which I don't necessarily prefer over yaml or json for declarative tasks but that's another debate). Most of the simplicity argument comes from the single server/worker binary and ease of deployment on the ops side, which if you use managed k8s like GKE might not mean all that much. If you're going to implement and run your own clusters that's a bigger deal. Looking at it from the consumer side and comparing like to like I don't feel its a _lot_ simpler. Task groups and pods, tasks and containers, config, networking, storage, it is all different on nomad but not, imo, tremendously simpler. That makes sense given the nature of the problem the systems are addressing. Over the last five years kubernetes has grown horizontally a great deal to encompass more and more enterprise features and use cases, and I feel in general this is a repeating pattern: a thing is created, it works well and becomes popular, more people use it and join the community bringing their own use cases, which drives the addition of features to the thing, which eventually gets complicated enough that people are attracted to a new thing that does largely the same stuff but hasn't had a lot of the new features added yet. Rinse and repeat. On a somewhat unrelated note: I'm not very attracted to server-side features for tracking and manipulating versions of deployments and their history. I'm a fan of git ops, and what I want out of the server-side runtime is reliable, fast reconciliation of the infrastructure state with the release branch in my repo. If I want to roll back I would much rather revert in git and redeploy. It seems troublesome and error prone to rely on a cli command to force the server side state to a previous iteration that is not in sync with what master in the repo says should be the canonical release version. Interested in others' thoughts, but maybe its a separate topic.

AnxiousCoward 6y ago
Having a rollback feature at the orchestrator level is useful when you want the rollback to be fast. Sometimes that's critical - when you deploy something buggy in production. One reason I'd prefer to use Nomad over K8s is scalability. K8s becomes slow and occasionally acts weird with just a few thousand pods. (Not that it is particularly fast with small numbers of pods.) Nomad is known to run reliably in production with many thousands of mixed workloads, not just containers. Another reason is its specialization. I'd rather deal with a handful of independent, well documented components (consul, vault, nomad, basically) than with one that does all things, but in a way I have to occasionally fight, and which, by necessity, given its breadth, is awfully documented. We do use k8s but still run vault for secrets - k8s secrets are a joke. We don't use consul because our discovery needs are simple, but the service abstraction in k8s is weak at best. While I'm not a big fan of HCL, any version, either, I do think that being able to manage everything, from cluster configuration to application deployment, via the same tool (terraform), in git, is more convenient than still having to use terraform for infrastructure but being forced to use helm on top of it - you might be able to maintain applications as terraform HCL scripts, but it would be unreasonable, given that for many applications that you'll want to deploy there are readymade charts available. From a strictly theoretical point of view, designing any kind of software the way kubernetes is designed is bad practice. It consists of very few components - but which do many different things. Some people work hard to maintain the system usable, but IMO the bad structure shows in how users need to interact with the system. Which is why, if the choice was mine, and not hype-driven, I'd take managed nomad with consul, vault, terraform and some git repo (my choice: gitlab, because it comes with a nice integrated CI) already integrated any time over managed kubernetes. But that's just me.

AnxiousCoward 6y ago

Having a rollback feature at the orchestrator level is useful when you want the rollback to be fast. Sometimes that's critical - when you deploy something buggy in production. One reason I'd prefer to use Nomad over K8s is scalability. K8s becomes slow and occasionally acts weird with just a few thousand pods. (Not that it is particularly fast with small numbers of pods.) Nomad is known to run reliably in production with many thousands of mixed workloads, not just containers. Another reason is its specialization. I'd rather deal with a handful of independent, well documented components (consul, vault, nomad, basically) than with one that does all things, but in a way I have to occasionally fight, and which, by necessity, given its breadth, is awfully documented. We do use k8s but still run vault for secrets - k8s secrets are a joke. We don't use consul because our discovery needs are simple, but the service abstraction in k8s is weak at best. While I'm not a big fan of HCL, any version, either, I do think that being able to manage everything, from cluster configuration to application deployment, via the same tool (terraform), in git, is more convenient than still having to use terraform for infrastructure but being forced to use helm on top of it - you might be able to maintain applications as terraform HCL scripts, but it would be unreasonable, given that for many applications that you'll want to deploy there are readymade charts available. From a strictly theoretical point of view, designing any kind of software the way kubernetes is designed is bad practice. It consists of very few components - but which do many different things. Some people work hard to maintain the system usable, but IMO the bad structure shows in how users need to interact with the system. Which is why, if the choice was mine, and not hype-driven, I'd take managed nomad with consul, vault, terraform and some git repo (my choice: gitlab, because it comes with a nice integrated CI) already integrated any time over managed kubernetes. But that's just me.