12 ms·

Sad to see this, but I will again shill for HashiCorp's Nomad as a better alternative to Kubernetes.

sepositus 2y ago
I agree up to a certain scale. I've managed a large Nomad/Consul setup (multiple clusters, geographically separated), and it was nothing but a nightmare. I believe fly.io had a similar experience.
film42 2y ago
Nomad is way easier to self-manage than K8s, but GCP does that for me, with all the compliance boxes checked, for extremely cheap. Every cloud provider is in that boat. Nomad will be more work and more money, be it compute or enterprise fees. I'm sticking with k8s.
- ericjmorey 2y ago
  The convenience and ease of use isn't worth the increased costs?
  - miningape 2y ago
    Who's to say it doesn't cost more in time and effort?
    - airplaneears 2y ago
      [flagged]
- airplaneears 2y ago
  [flagged]
lovehashbrowns 2y ago
Definitely not better than Kubernetes, but I don't regret working on it and I like it as a simpler alternative to Kubernetes. I remember trying to hire people for it and not a single person ever even heard of it.
- ForHackernews 2y ago
  > I remember trying to hire people for it and not a single person ever even heard of it. I know, it's really sad. Kubernetes won because of mindshare and hype and 500,000 CNCF consulting firms selling their own rubbish to "finally make k8s easy to use".
  - lovehashbrowns 2y ago
    It's also sad how much Hashicorp apparently ignores marketing for it. Especially when it works with Consul which is already in a lot of places. Also another funny memory is going into a decomm meeting where I was gonna ignore it like usual since 99% of it was another team decommissioning on-prem stuff. And out of nowhere I hear "we just got approval to decomm Consul. We'll be shutting down the servers in two weeks and waiting to see if anything breaks.." followed by my very stern "NO YOU WILL NOT" ahaha. What a mess that company was
cowgoesmoo 2y ago
Having worked with a very large Nomad cluster, I cannot disagree more. For simple use cases, sure, but you could also just use AWS ECS or a similar cloud tool for an even easier experience.
- cmckn 2y ago
  Can you quantify “very large”?
  - cowgoesmoo 2y ago
    20k+ nodes and 200k+ allocs. To be fair, Kubernetes cannot support this large of a cluster. Most of my issues with it aren't related to the scale though. I wasn't involved in the operations of the cluster (though I did hear many "fun" stories from that team), I was just a user of Nomad trying to run a few thousand stateful allocs. Without custom resources and custom controllers, managing stateful services was a pain in the ass. Critical bugs would also often take years to get fixed. I had lots of fun getting paged in the middle of the night because 2 allocs would suddenly decide they now have the same index (https://github.com/hashicorp/nomad/issues/10727 https://github.com/hashicorp/nomad/issues/10727)
airplaneears 2y ago
I vastly prefer a hole in the head, personally.

I agree up to a certain scale. I've managed a large Nomad/Consul setup (multiple clusters, geographically separated), and it was nothing but a nightmare. I believe fly.io had a similar experience.

film42 2y ago

Nomad is way easier to self-manage than K8s, but GCP does that for me, with all the compliance boxes checked, for extremely cheap. Every cloud provider is in that boat. Nomad will be more work and more money, be it compute or enterprise fees. I'm sticking with k8s.

ericjmorey 2y ago
The convenience and ease of use isn't worth the increased costs?
- miningape 2y ago
  Who's to say it doesn't cost more in time and effort?
  - airplaneears 2y ago
    [flagged]
airplaneears 2y ago
[flagged]

lovehashbrowns 2y ago

Definitely not better than Kubernetes, but I don't regret working on it and I like it as a simpler alternative to Kubernetes. I remember trying to hire people for it and not a single person ever even heard of it.

ForHackernews 2y ago
> I remember trying to hire people for it and not a single person ever even heard of it. I know, it's really sad. Kubernetes won because of mindshare and hype and 500,000 CNCF consulting firms selling their own rubbish to "finally make k8s easy to use".
- lovehashbrowns 2y ago
  It's also sad how much Hashicorp apparently ignores marketing for it. Especially when it works with Consul which is already in a lot of places. Also another funny memory is going into a decomm meeting where I was gonna ignore it like usual since 99% of it was another team decommissioning on-prem stuff. And out of nowhere I hear "we just got approval to decomm Consul. We'll be shutting down the servers in two weeks and waiting to see if anything breaks.." followed by my very stern "NO YOU WILL NOT" ahaha. What a mess that company was

cowgoesmoo 2y ago

Having worked with a very large Nomad cluster, I cannot disagree more. For simple use cases, sure, but you could also just use AWS ECS or a similar cloud tool for an even easier experience.

cmckn 2y ago
Can you quantify “very large”?
- cowgoesmoo 2y ago
  20k+ nodes and 200k+ allocs. To be fair, Kubernetes cannot support this large of a cluster. Most of my issues with it aren't related to the scale though. I wasn't involved in the operations of the cluster (though I did hear many "fun" stories from that team), I was just a user of Nomad trying to run a few thousand stateful allocs. Without custom resources and custom controllers, managing stateful services was a pain in the ass. Critical bugs would also often take years to get fixed. I had lots of fun getting paged in the middle of the night because 2 allocs would suddenly decide they now have the same index (https://github.com/hashicorp/nomad/issues/10727 https://github.com/hashicorp/nomad/issues/10727)

airplaneears 2y ago

I vastly prefer a hole in the head, personally.