4 ms·

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no

by SEJeff 11y ago

I'm genuinely not sure why they are trying to compete with the likes of Mesos or Kubernetes, or what they are really trying to achieve here. There is simply no way they'll build a community around nomad 1/2 as big as either of the aforementioned even if the software is really good.

fapjacks 11y ago
I'm sure to a fan that "really freaking loves Mesos" you can't imagine other software competing with Mesos, but Hashicorp writes excellent software generally. Serf and Consul have completely sold me on Nomad, and I haven't even used it yet. I think there will be similar draw for many people that have used Hashicorp's software before.
- SEJeff 11y ago
  No it isn't about koolaid, it is about technology. Consul, Vagrant, and Vault are all excellent Hashicorp technologies I've used. They are great, and some of the best in the industry for their problem spaces. You've got Mesos scaling to 10,000+ physical node clusters today in production at the likes of companies like Apple and Twitter. You've got Kubernetes being adopted and developed by pretty much all of the open source heavyweights and it came from the experienced Google developed building... Google. Kubernetes ontop of Mesos is basically the holy grail in my personal opinion where you get the best Ops (mesos) story mixed with the best Dev (k8s) story. I guess we'll see how much Nomad takes off :) Hashicorp isn't a huge company, it seems like to me their best bet is keeping the focus relatively small so they can be the best at what they do. Even if Nomad is a huge hit and is amazing, it still seems kind of sad that they couldn't simply double down and help out with kubernetes. I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of.
  - Rapzid 11y ago
    I'm sort of surprised to hear this view of the situation which is about the complete opposite of what I've had for the past few years. I have felt it's a shame that hashicorp have created consul, vault, a new raft implementation (which they got tons of flack for then etcd made THEIR OWN re-write which fixed a tons of issues this year..), serf, etc., and nobody has adopted or built on them. K8's current secrets solution is a bit underwhelming TBH, as an example. Kubernetes has support for 250 node clusters in their list of blockers: https://github.com/kubernetes/kubernetes/blob/master/docs/roadmap.md#blocking-features https://github.com/kubernetes/kubernetes/blob/master/docs/ro...
- ninkendo 11y ago
  It's just sad that there's yet another player that wants to be at the very bottom of the clustering stack. For physical resource management (memory and CPUs, etc), by definition you have to pick one solution and stick with it (because you don't want multiple cluster managers thinking they exclusively own the same hardware.) Mesos is the only player right now that's trying to do multitenancy; separating the concerns of resource management from scheduler algorithms, UI's, etc. You put mesos on your hardware as your datacenter kernel, then you play around with various schedulers to support different workloads. Done properly, I'd much rather see Nomad exist as a Mesos framework, so I could colocate it along side chronos/spark/marathon and see how it fits in. Instead if I wanted to run this I have to partition up my hardware and dedicate some to it, which is the whole thing that cluster managers are supposed to save you from having to do (by running multiple workloads on the same hardware.) I get that this is operationally simpler to set up and install, but with that simplicity comes a huge amount of lock-in, because you can't do anything else with your hardware other than run Nomad on it.
  - fapjacks 11y ago
    Yeah, it feels almost like the mess of JavaScript libraries of the last five years: Everybody is dropping a hook into the water with some newfangled containerization technology, hoping to make it big.
atombender 11y ago
This is a space where you can compete on operational complexity. Mesos and Kubernetes are fairly complicated to set up and run, with dependencies and intricacies that Nomad just doesn't have. From your other comment: > I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of. Being for microservices doesn't mean you should be a microservice. Kubernetes your control plane involves 5 services (Kubelet, proxy, Docker, replication controller, etcd) that need to be up and running before you even have started an app. Then the question is what happens to your system if one or two of those go down, or if they become bottlenecks, or need to be upgraded, and so on. Having evaluated both Mesos and Kubernetes, Nomad is a lot more attractive to me due to its simplicity and "turnkey" approach.
- SEJeff 11y ago
  When they become bottlenecks, you scale them out. That is the beauty of the design, vs a monolith like nomad :) It does win on setup for sure. This isn't an easy problem to solve however.
  - lobster_johnson 11y ago
    Not all bottlenecks scale linearly (and "bottleneck" was just one of several words I used). I'm sure Kubernetes is well-designed and battle-tested, but to most developers these are black boxes that need to be studied and learned, each with its own set of complexities and workarounds and warts. And so on. Moving parts are moving parts no matter how well they are designed. More of them always add complexity, by definition; they never reduce it.

by SEJeff 11y ago

fapjacks 11y ago

I'm sure to a fan that "really freaking loves Mesos" you can't imagine other software competing with Mesos, but Hashicorp writes excellent software generally. Serf and Consul have completely sold me on Nomad, and I haven't even used it yet. I think there will be similar draw for many people that have used Hashicorp's software before.

SEJeff 11y ago
No it isn't about koolaid, it is about technology. Consul, Vagrant, and Vault are all excellent Hashicorp technologies I've used. They are great, and some of the best in the industry for their problem spaces. You've got Mesos scaling to 10,000+ physical node clusters today in production at the likes of companies like Apple and Twitter. You've got Kubernetes being adopted and developed by pretty much all of the open source heavyweights and it came from the experienced Google developed building... Google. Kubernetes ontop of Mesos is basically the holy grail in my personal opinion where you get the best Ops (mesos) story mixed with the best Dev (k8s) story. I guess we'll see how much Nomad takes off :) Hashicorp isn't a huge company, it seems like to me their best bet is keeping the focus relatively small so they can be the best at what they do. Even if Nomad is a huge hit and is amazing, it still seems kind of sad that they couldn't simply double down and help out with kubernetes. I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of.
- Rapzid 11y ago
  I'm sort of surprised to hear this view of the situation which is about the complete opposite of what I've had for the past few years. I have felt it's a shame that hashicorp have created consul, vault, a new raft implementation (which they got tons of flack for then etcd made THEIR OWN re-write which fixed a tons of issues this year..), serf, etc., and nobody has adopted or built on them. K8's current secrets solution is a bit underwhelming TBH, as an example. Kubernetes has support for 250 node clusters in their list of blockers: https://github.com/kubernetes/kubernetes/blob/master/docs/roadmap.md#blocking-features https://github.com/kubernetes/kubernetes/blob/master/docs/ro...
ninkendo 11y ago
It's just sad that there's yet another player that wants to be at the very bottom of the clustering stack. For physical resource management (memory and CPUs, etc), by definition you have to pick one solution and stick with it (because you don't want multiple cluster managers thinking they exclusively own the same hardware.) Mesos is the only player right now that's trying to do multitenancy; separating the concerns of resource management from scheduler algorithms, UI's, etc. You put mesos on your hardware as your datacenter kernel, then you play around with various schedulers to support different workloads. Done properly, I'd much rather see Nomad exist as a Mesos framework, so I could colocate it along side chronos/spark/marathon and see how it fits in. Instead if I wanted to run this I have to partition up my hardware and dedicate some to it, which is the whole thing that cluster managers are supposed to save you from having to do (by running multiple workloads on the same hardware.) I get that this is operationally simpler to set up and install, but with that simplicity comes a huge amount of lock-in, because you can't do anything else with your hardware other than run Nomad on it.
- fapjacks 11y ago
  Yeah, it feels almost like the mess of JavaScript libraries of the last five years: Everybody is dropping a hook into the water with some newfangled containerization technology, hoping to make it big.

atombender 11y ago

This is a space where you can compete on operational complexity. Mesos and Kubernetes are fairly complicated to set up and run, with dependencies and intricacies that Nomad just doesn't have. From your other comment: > I do find it ironic that they talk about how nomad is for microservices and then make a dig at the several microservices that k8s is made up of. Being for microservices doesn't mean you should be a microservice. Kubernetes your control plane involves 5 services (Kubelet, proxy, Docker, replication controller, etcd) that need to be up and running before you even have started an app. Then the question is what happens to your system if one or two of those go down, or if they become bottlenecks, or need to be upgraded, and so on. Having evaluated both Mesos and Kubernetes, Nomad is a lot more attractive to me due to its simplicity and "turnkey" approach.

SEJeff 11y ago
When they become bottlenecks, you scale them out. That is the beauty of the design, vs a monolith like nomad :) It does win on setup for sure. This isn't an easy problem to solve however.
- lobster_johnson 11y ago
  Not all bottlenecks scale linearly (and "bottleneck" was just one of several words I used). I'm sure Kubernetes is well-designed and battle-tested, but to most developers these are black boxes that need to be studied and learned, each with its own set of complexities and workarounds and warts. And so on. Moving parts are moving parts no matter how well they are designed. More of them always add complexity, by definition; they never reduce it.