5 ms·

sigh I've almost given up nudging folks on here to give Nomad a spin. It's so easy to deploy and play with, heck you can run a client and server with a single g

by heipei 6y ago

sigh I've almost given up nudging folks on here to give Nomad a spin. It's so easy to deploy and play with, heck you can run a client and server with a single go binary on your local machine. If you compare that to all the hundreds of different (and soon-to-be-deprecated) installation instructions for k8s, k8s feels like a joke at best, or an evil plot at worst to get everyone to use hosted k8s. Honestly, people use hosted k8s because you need a full-time person/team to run it, so in the end, even though k8s is Open Source, Google (and AWS) still win... Nevermind that Nomad is fast, easy to grasp, very concise and can run non-Docker workloads. Oh well...

jordanbeiber 6y ago
The “evil plot” is what I gather. K8s have it’s bright spots, but it can’t touch the flexibility of nomad & consul. Now that nomad/consul have autoscaling and csi as well as native envoy integration the gap, with regards to container workloads, have shrunk quite a bit!
klohto 6y ago
I'm interested to hear more! The Nomad page gives me a good overview but doesn't tell me WHY should I migrate from an existing k8 cluster. Would you be able to sell it to me?
- jordanbeiber 6y ago
  Off the top of my head: - Nomad is a more general type of workload scheduler. What I mean with "general" is that you can schedule pretty much anything over a bunch of hosts - not just containers e.g you have a JRE on the node? Just schedule your jar. Execute a raw shell command? Not a problem. - Batch jobs with parameters? Just send it to the "dispatch" endpoint with parameters and an optional payload. Often can replace "serverless" and sometimes work queues. - Consul is a "general purpose" service discovery (dns & distributed K/V) that can span over all your services, not just k8s ones (for this very reason Hashicorp have built a k8s -> consul services sync tool: it is often needed!) You include consul agent with a service definition in roles on your ansible/salt/puppet managed VM's. They can be included in the mesh, and SD works seamlessly between hosts and containers. - The consul connect (mesh) feature with envoy is much more comprehensible than istio. I use to say that nomad + consul is more in-line with the "unix philosophy". You have these two binaries that do what they do independently, but they can be hooked together if you would like to use distributed K/V and service discovery for your scheduled workload. Add Vault to that - which often is used without consul & nomad, but integrates as well. They can all be easily integrated with through well defined rest API's and do not have to be consumed as a big black box. K8s is all or nothing.
  - klohto 6y ago
    Thanks a lot. I will definitely give it a try. Thankfully, our services are very easily manageable and provide clear Rest APIs, it's just the overhead cluster that's making us slow. I would love to have something like Nomad ready and available for single-tenant deployments of our service. It's where you need to tweak a lot and do it fast.
blondin 6y ago
my thing with hashicorp products is the feeling that they are geared towards enterprise and not the community at large. people don't write about them or teach you about them. i have only seen a couple of youtube videos, just mentioning them. i got a book on terraform (the only one i could find that covers it) that wanted me to use AWS to even begin the examples. it feels like these tools are only used in the enterprise realm. very sad and discouraging.

by heipei 6y ago

jordanbeiber 6y ago

The “evil plot” is what I gather. K8s have it’s bright spots, but it can’t touch the flexibility of nomad & consul. Now that nomad/consul have autoscaling and csi as well as native envoy integration the gap, with regards to container workloads, have shrunk quite a bit!

klohto 6y ago

I'm interested to hear more! The Nomad page gives me a good overview but doesn't tell me WHY should I migrate from an existing k8 cluster. Would you be able to sell it to me?

jordanbeiber 6y ago
Off the top of my head: - Nomad is a more general type of workload scheduler. What I mean with "general" is that you can schedule pretty much anything over a bunch of hosts - not just containers e.g you have a JRE on the node? Just schedule your jar. Execute a raw shell command? Not a problem. - Batch jobs with parameters? Just send it to the "dispatch" endpoint with parameters and an optional payload. Often can replace "serverless" and sometimes work queues. - Consul is a "general purpose" service discovery (dns & distributed K/V) that can span over all your services, not just k8s ones (for this very reason Hashicorp have built a k8s -> consul services sync tool: it is often needed!) You include consul agent with a service definition in roles on your ansible/salt/puppet managed VM's. They can be included in the mesh, and SD works seamlessly between hosts and containers. - The consul connect (mesh) feature with envoy is much more comprehensible than istio. I use to say that nomad + consul is more in-line with the "unix philosophy". You have these two binaries that do what they do independently, but they can be hooked together if you would like to use distributed K/V and service discovery for your scheduled workload. Add Vault to that - which often is used without consul & nomad, but integrates as well. They can all be easily integrated with through well defined rest API's and do not have to be consumed as a big black box. K8s is all or nothing.
- klohto 6y ago
  Thanks a lot. I will definitely give it a try. Thankfully, our services are very easily manageable and provide clear Rest APIs, it's just the overhead cluster that's making us slow. I would love to have something like Nomad ready and available for single-tenant deployments of our service. It's where you need to tweak a lot and do it fast.

blondin 6y ago

my thing with hashicorp products is the feeling that they are geared towards enterprise and not the community at large. people don't write about them or teach you about them. i have only seen a couple of youtube videos, just mentioning them. i got a book on terraform (the only one i could find that covers it) that wanted me to use AWS to even begin the examples. it feels like these tools are only used in the enterprise realm. very sad and discouraging.