4 ms·

At the core of K8s is an Etcd cluster, so if you’re spinning your own it’s the same thing. However, part of the value-add of cloud providers in the K8s space is

At the core of K8s is an Etcd cluster, so if you’re spinning your own it’s the same thing. However, part of the value-add of cloud providers in the K8s space is that they hide it from you.

figassis 4y ago
I understand that, but I already invested the time to work with etcd and can handle a few of k8s failure modes. Really do not want to throw that away for something that has a similar dependency, and that I do not now know well. Nomad might be simpler to use, but not to setup or manage due to Consul dependency. Also, a lot of these products are missing a lot of documentation. I often find the tutorials are inflexible and focus on only 2 use cases out of the 4 most common. For example, focus on cloud when lots of engineers would rather deploy on bare metal. I would love for there so be one simple Nomad ansible role that would help setup nomad on bare metal with wireguard networking. This would probably be easy if consul was not required and it did not need so much babysitting.
- sofixa 4y ago
  Just throwing this out there: Since Nomad 1.3, Consul is no longer required for Service Discovery, and since 1.4 Vault will no longer be needed for KV secrets. And for what it's worth, i used to run Nomad+Consul in production on bare metal and it was easy to setup (2x 3 servers with a binary to start with some basic conf) and maintain (an update every month or so, which consisted of upgrading the binary on each node and restarting the service after 5 mins of reading the release notes).
metadat 4y ago
Etcd has come a long way and generally doesn't behave strangely or cause big issues these days. The cloud providers generally aren't actually doing any significant amount of work on your behalf on the etcd front.
- tuananh 4y ago
  this happened very recently Etcd v3.5.[0-2] is not recommended for production => https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQc/m/rSgNOzV2BwAJ https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQ... https://github.com/etcd-io/etcd/issues/13766#issuecomment-1078897588 https://github.com/etcd-io/etcd/issues/13766#issuecomment-10...

At the core of K8s is an Etcd cluster, so if you’re spinning your own it’s the same thing. However, part of the value-add of cloud providers in the K8s space is that they hide it from you.

figassis 4y ago

I understand that, but I already invested the time to work with etcd and can handle a few of k8s failure modes. Really do not want to throw that away for something that has a similar dependency, and that I do not now know well. Nomad might be simpler to use, but not to setup or manage due to Consul dependency. Also, a lot of these products are missing a lot of documentation. I often find the tutorials are inflexible and focus on only 2 use cases out of the 4 most common. For example, focus on cloud when lots of engineers would rather deploy on bare metal. I would love for there so be one simple Nomad ansible role that would help setup nomad on bare metal with wireguard networking. This would probably be easy if consul was not required and it did not need so much babysitting.

sofixa 4y ago
Just throwing this out there: Since Nomad 1.3, Consul is no longer required for Service Discovery, and since 1.4 Vault will no longer be needed for KV secrets. And for what it's worth, i used to run Nomad+Consul in production on bare metal and it was easy to setup (2x 3 servers with a binary to start with some basic conf) and maintain (an update every month or so, which consisted of upgrading the binary on each node and restarting the service after 5 mins of reading the release notes).

metadat 4y ago

Etcd has come a long way and generally doesn't behave strangely or cause big issues these days. The cloud providers generally aren't actually doing any significant amount of work on your behalf on the etcd front.

tuananh 4y ago
this happened very recently Etcd v3.5.[0-2] is not recommended for production => https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQc/m/rSgNOzV2BwAJ https://groups.google.com/a/kubernetes.io/g/dev/c/B7gJs88XtQ... https://github.com/etcd-io/etcd/issues/13766#issuecomment-1078897588 https://github.com/etcd-io/etcd/issues/13766#issuecomment-10...