4 ms·

This post should probably have (2019) in the title. I'm the Nomad Team Lead and can try to answer any questions. Since this post was made the team has expanded

This post should probably have (2019) in the title. I'm the Nomad Team Lead and can try to answer any questions. Since this post was made the team has expanded, and the task/job restarting issue they link has been (mostly) addressed. Also new since this post is our Consul Connect integration which can accomplish similar goals to k8s network policies, albeit opt in and with the actual discovery/networking code living in Consul/Envoy respectively.

scarface74 7y ago
I absolutely loved Nomad when I used it. We were stuck with both Windows, legacy apps and command line scripts and no time to learn Docker or anything like K8s. We were already using Consul so bringing in Nomad to schedule everything - batch files, .Net executables, some legacy programs, etc was a godsend. Unfortunately, as much as I loved the Nomad+Consul combination, I really couldn’t suggest it today. It is so much easier to find qualified K8s people than Nomad+Consul people I couldn’t in good conscience recommend it. But this is all a moot point to me. While if I were leading another on prem project I would use K8s. We are all in on AWS+ECS+Fargate where I work now and we really don’t care about the lock in boogie man. Given a choice, I would still say at least if you’re on AWS, use the native offerings. The value of hypothetically being able to migrate a large infrastructure “seamlessly” is vastly overrated.
- schmichael 7y ago
  > I absolutely loved Nomad when I used it. Glad to hear it! > Unfortunately, as much as I loved the Nomad+Consul combination, I really couldn’t suggest it today. This is a fair critique and a problem any project living in a world defined by a strong incumbent suffers. You made me realize we need resources to help k8s users translate their existing knowledge to Nomad as many people looking at Nomad will have k8s experience these days. So thanks for this comment. Maybe with the right docs/resources we can at least minimize the very real cost of using a non-dominant tool. > But this is all a moot point to me. We are all in on AWS+ECS+Fargate where I work now and we really don’t care about the lock in boogie man. This was me in a past lives/jobs! HashiCorp's entire product line (with the exception of maybe Packer and Vagrant) become much more compelling for multi-cloud, hybrid-cloud, and on-prem orgs.
  - scarface74 7y ago
    In my world, there are two types of “architects” and “consultants”. You have the kind that are “Smart People” (tm) and tell their clients that “this is the way that it should be done” and you have those that “will meet the clients where they are”. From my experience with Nomad and the little I know about K8s, Nomad is the latter. If you can run it from the command line, you can run it with Nomad. This in and of itself is a great value proposition. But, Nomad does have the disadvantage, I posted about above and has to fight the “no one ever got fired for buying IBM”. I was able to get buy in only after I told the CTO, “it’s made by the same people who make Consul and Terraform”.
jugg1es 7y ago
Kudos to your team. We have had great success with Nomad/Consul/Fabio. My only issue has been that there isn't a lot of insight into what is going on with Fabio. There aren't really any configuration options. I admit we are a year(plus) behind on versions so maybe things have changed.
- schmichael 7y ago
  > My only issue has been that there isn't a lot of insight into what is going on with Fabio. There aren't really any configuration options. We have load balancing docs for 4 of the most popular load balancers including Fabio: https://www.nomadproject.io/guides/load-balancing/load-balancing.html https://www.nomadproject.io/guides/load-balancing/load-balan... We're monitoring Frank's search for a maintainer but unfortunately don't have the resources to commit to it at the moment. If Fabio becomes abandoned and we're still unable to take over we'll remove it from our docs. Please feel free to open an issue if you have a specific idea or question. We triage every issue and appreciate user feedback immensely!
  - jugg1es 7y ago
    Wow we are really behind. I did not realize that Fabio was in danger of being abandoned. Do you have a recommendation on what we should be doing? I guess we can move to nginx as we have experience with that. Fabio was really nice cuz it just worked. You guys really can't spare the resources to pick up such a necessary part of a container orchestration system? I guess I can see it both ways, where you think that that domain already has very good solutions and you don't want to waste time reinventing the wheel. But then it's much easier for people to pick up your stack and decide to use it when it has load balancing automatically included. Just saying.
    - schmichael 7y ago
      Yeah there's definitely lots of internal discussion to see what can be done. Traefik is getting quite popular and is arguably the heir apparent. Nginx and HAProxy also work and are as battle tested as software comes. We may also add a first class notion of ingress load balancing in the future and Envoy would be the natural choice there because we already use it for Nomad's Consul Connect integration.
peterwwillis 7y ago
In the past I tried to follow the Nomad+Consul set-up guides, but on my local machine rather than 3 separate machines, and I found it nearly impossible to get everything running. I ran into a dozen different errors and it seemed to be a lot more haphazard than the docs suggested. I really wanted to get into it, but that experience soured me on the idea that it would be a "simple" solution after all. (I'm willing to dig up and replicate my findings if you're interested)
- schmichael 7y ago
  I'm sorry to hear that! We definitely want to improve our cross-product bootstrapping situation. Running "nomad agent -dev" and "consul agent -dev" on your local machine in 2 different terminals should work. Running more than one Nomad agent on the same machine is not recommended as you have to carefully configure ports and paths not to conflict. There are some demo clusters available via Vagrant or Terraform, but we should really do a better job of going from the dev agent to a cluster: - https://github.com/hashicorp/nomad/tree/master/demo https://github.com/hashicorp/nomad/tree/master/demo - https://github.com/hashicorp/nomad/tree/master/terraform https://github.com/hashicorp/nomad/tree/master/terraform

scarface74 7y ago

I absolutely loved Nomad when I used it. We were stuck with both Windows, legacy apps and command line scripts and no time to learn Docker or anything like K8s. We were already using Consul so bringing in Nomad to schedule everything - batch files, .Net executables, some legacy programs, etc was a godsend. Unfortunately, as much as I loved the Nomad+Consul combination, I really couldn’t suggest it today. It is so much easier to find qualified K8s people than Nomad+Consul people I couldn’t in good conscience recommend it. But this is all a moot point to me. While if I were leading another on prem project I would use K8s. We are all in on AWS+ECS+Fargate where I work now and we really don’t care about the lock in boogie man. Given a choice, I would still say at least if you’re on AWS, use the native offerings. The value of hypothetically being able to migrate a large infrastructure “seamlessly” is vastly overrated.

schmichael 7y ago
> I absolutely loved Nomad when I used it. Glad to hear it! > Unfortunately, as much as I loved the Nomad+Consul combination, I really couldn’t suggest it today. This is a fair critique and a problem any project living in a world defined by a strong incumbent suffers. You made me realize we need resources to help k8s users translate their existing knowledge to Nomad as many people looking at Nomad will have k8s experience these days. So thanks for this comment. Maybe with the right docs/resources we can at least minimize the very real cost of using a non-dominant tool. > But this is all a moot point to me. We are all in on AWS+ECS+Fargate where I work now and we really don’t care about the lock in boogie man. This was me in a past lives/jobs! HashiCorp's entire product line (with the exception of maybe Packer and Vagrant) become much more compelling for multi-cloud, hybrid-cloud, and on-prem orgs.
- scarface74 7y ago
  In my world, there are two types of “architects” and “consultants”. You have the kind that are “Smart People” (tm) and tell their clients that “this is the way that it should be done” and you have those that “will meet the clients where they are”. From my experience with Nomad and the little I know about K8s, Nomad is the latter. If you can run it from the command line, you can run it with Nomad. This in and of itself is a great value proposition. But, Nomad does have the disadvantage, I posted about above and has to fight the “no one ever got fired for buying IBM”. I was able to get buy in only after I told the CTO, “it’s made by the same people who make Consul and Terraform”.

jugg1es 7y ago

Kudos to your team. We have had great success with Nomad/Consul/Fabio. My only issue has been that there isn't a lot of insight into what is going on with Fabio. There aren't really any configuration options. I admit we are a year(plus) behind on versions so maybe things have changed.

schmichael 7y ago
> My only issue has been that there isn't a lot of insight into what is going on with Fabio. There aren't really any configuration options. We have load balancing docs for 4 of the most popular load balancers including Fabio: https://www.nomadproject.io/guides/load-balancing/load-balancing.html https://www.nomadproject.io/guides/load-balancing/load-balan... We're monitoring Frank's search for a maintainer but unfortunately don't have the resources to commit to it at the moment. If Fabio becomes abandoned and we're still unable to take over we'll remove it from our docs. Please feel free to open an issue if you have a specific idea or question. We triage every issue and appreciate user feedback immensely!
- jugg1es 7y ago
  Wow we are really behind. I did not realize that Fabio was in danger of being abandoned. Do you have a recommendation on what we should be doing? I guess we can move to nginx as we have experience with that. Fabio was really nice cuz it just worked. You guys really can't spare the resources to pick up such a necessary part of a container orchestration system? I guess I can see it both ways, where you think that that domain already has very good solutions and you don't want to waste time reinventing the wheel. But then it's much easier for people to pick up your stack and decide to use it when it has load balancing automatically included. Just saying.
  - schmichael 7y ago
    Yeah there's definitely lots of internal discussion to see what can be done. Traefik is getting quite popular and is arguably the heir apparent. Nginx and HAProxy also work and are as battle tested as software comes. We may also add a first class notion of ingress load balancing in the future and Envoy would be the natural choice there because we already use it for Nomad's Consul Connect integration.

peterwwillis 7y ago

In the past I tried to follow the Nomad+Consul set-up guides, but on my local machine rather than 3 separate machines, and I found it nearly impossible to get everything running. I ran into a dozen different errors and it seemed to be a lot more haphazard than the docs suggested. I really wanted to get into it, but that experience soured me on the idea that it would be a "simple" solution after all. (I'm willing to dig up and replicate my findings if you're interested)

schmichael 7y ago
I'm sorry to hear that! We definitely want to improve our cross-product bootstrapping situation. Running "nomad agent -dev" and "consul agent -dev" on your local machine in 2 different terminals should work. Running more than one Nomad agent on the same machine is not recommended as you have to carefully configure ports and paths not to conflict. There are some demo clusters available via Vagrant or Terraform, but we should really do a better job of going from the dev agent to a cluster: - https://github.com/hashicorp/nomad/tree/master/demo https://github.com/hashicorp/nomad/tree/master/demo - https://github.com/hashicorp/nomad/tree/master/terraform https://github.com/hashicorp/nomad/tree/master/terraform