5 ms·

Disclaimer: I'm the HashiCorp Nomad Engineering Team Lead This may be my favorite article on the subject, and I've read a lot! Others have suggested some minor

Disclaimer: I'm the HashiCorp Nomad Engineering Team Lead This may be my favorite article on the subject, and I've read a lot! Others have suggested some minor corrections but overall you've done a fantastic job accurately and fairly portraying Nomad as far as I'm concerned. I really appreciate the final paragraph as I constantly field questions about whether or not a k8s-only world is a foregone conclusion. I think there's room for more than one option, and I'm glad you noticed HashiCorp thinks so too!

jordemort 4y ago
Thanks, I'm glad you liked it! I've used Nomad a bit at a previous job; due to its simplicity, we were much more comfortable embedding it in the virtual appliance images we were building. Nomad meant we had to rewrite the k8s configs that the hosted version of the app was using, but even with one of the all-in-one k8s distros, we were worried about shipping that many hidden moving parts.
jerrac 4y ago
What options are there for managing persistent storage in Nomad? I've been through the very basic initial getting started tutorial, but haven't had a chance to dig any deeper. When I was trying kube, I found Longhorn to be the one thing I liked. Is there anything like that for Nomad? I've been trying to get a MariaDB Galera cluster running on Docker Swarm for the past couple weeks and have been somewhat stymied a few things. Something like Longhorn would help a lot.
- schmichael 4y ago
  Nomad supports CSI like Kubernetes. Unfortunately many CSI storage providers assume kubernetes or adhere to kubernetes behavior even when it deviates from the spec, so there are gotchas. That being said they’re often glaring. So please report an issue to Nomad and the upstream of you hit an issue! Nomad has 2 other storage features: 1. Host volumes where nodes can advertise a volume by name and jobs can request being placed on nodes with that volume available. I think a lot of folks run databases this way so they can statically define their database nodes and not worry about the numerous runtime points of failure CSI introduces. 2. Ephemeral disk stickiness and migration: during deployments an instance of a job’s intrinsic local storage can be set to either get reused on the same node (sticky) or migrated to wherever the new instance is placed. Ephemeral disks is the oldest storage solution and is only a best effort (unlike CSI which may be able to reliably reattach and existing volume to a new node in the case of node failure). However I’ve heard of a lot of people happily using it for already distributed databases like Elasticsearch or Cassandra where the database can tolerate nomad’s best effort failing. (Sorry for lack of links as I’m on mobile. Googling the key phrases you care about should get you to our docs quickly and easily)
  - jerrac 4y ago
    Thanks! that gives a really good start.
andrenth 4y ago
Does nomad have a generic TCP/UDP load balancer (i.e. non HTTP) for inbound traffic? Something based on IPVS (akin to Kube-Router) would be ideal for the use-cases where Nomad would make sense for me.
- schmichael 4y ago
  Not builtin. We recommend folks run their load balancer in Nomad. Traefik is a common solution here as it already supports Nomad's service discovery. Other solutions often use templating and sig-hupping to configure: https://www.nomadproject.io/docs/job-specification/template https://www.nomadproject.io/docs/job-specification/template Nomad also supports CNI. It's not uncommon for folks to run their network's control plane as a Nomad job and use a CNI plugin to integrate their other Nomad jobs with it. This sort of approach allows for running multiple logical networks within a single Nomad cluster (eg perhaps segments of your cluster use a service mesh while data intensive or legacy applications use host networking).
  - andrenth 4y ago
    Cool. Something based on ipvs would be awesome though, for the low overhead load balancing and protocol independence.

jordemort 4y ago

Thanks, I'm glad you liked it! I've used Nomad a bit at a previous job; due to its simplicity, we were much more comfortable embedding it in the virtual appliance images we were building. Nomad meant we had to rewrite the k8s configs that the hosted version of the app was using, but even with one of the all-in-one k8s distros, we were worried about shipping that many hidden moving parts.

jerrac 4y ago

What options are there for managing persistent storage in Nomad? I've been through the very basic initial getting started tutorial, but haven't had a chance to dig any deeper. When I was trying kube, I found Longhorn to be the one thing I liked. Is there anything like that for Nomad? I've been trying to get a MariaDB Galera cluster running on Docker Swarm for the past couple weeks and have been somewhat stymied a few things. Something like Longhorn would help a lot.

schmichael 4y ago
Nomad supports CSI like Kubernetes. Unfortunately many CSI storage providers assume kubernetes or adhere to kubernetes behavior even when it deviates from the spec, so there are gotchas. That being said they’re often glaring. So please report an issue to Nomad and the upstream of you hit an issue! Nomad has 2 other storage features: 1. Host volumes where nodes can advertise a volume by name and jobs can request being placed on nodes with that volume available. I think a lot of folks run databases this way so they can statically define their database nodes and not worry about the numerous runtime points of failure CSI introduces. 2. Ephemeral disk stickiness and migration: during deployments an instance of a job’s intrinsic local storage can be set to either get reused on the same node (sticky) or migrated to wherever the new instance is placed. Ephemeral disks is the oldest storage solution and is only a best effort (unlike CSI which may be able to reliably reattach and existing volume to a new node in the case of node failure). However I’ve heard of a lot of people happily using it for already distributed databases like Elasticsearch or Cassandra where the database can tolerate nomad’s best effort failing. (Sorry for lack of links as I’m on mobile. Googling the key phrases you care about should get you to our docs quickly and easily)
- jerrac 4y ago
  Thanks! that gives a really good start.

andrenth 4y ago

Does nomad have a generic TCP/UDP load balancer (i.e. non HTTP) for inbound traffic? Something based on IPVS (akin to Kube-Router) would be ideal for the use-cases where Nomad would make sense for me.

schmichael 4y ago
Not builtin. We recommend folks run their load balancer in Nomad. Traefik is a common solution here as it already supports Nomad's service discovery. Other solutions often use templating and sig-hupping to configure: https://www.nomadproject.io/docs/job-specification/template https://www.nomadproject.io/docs/job-specification/template Nomad also supports CNI. It's not uncommon for folks to run their network's control plane as a Nomad job and use a CNI plugin to integrate their other Nomad jobs with it. This sort of approach allows for running multiple logical networks within a single Nomad cluster (eg perhaps segments of your cluster use a service mesh while data intensive or legacy applications use host networking).
- andrenth 4y ago
  Cool. Something based on ipvs would be awesome though, for the low overhead load balancing and protocol independence.