5 ms·

Nomad can do everything that K8s can

36 points by madmax108 3y ago

fesc 3y ago
I stopped reading when the solution for DNS was AWS and the solution for stateful sets was AWS EC2. This is just a very weird comparison. Seems like the author doesn’t understand why people run K8s. Of course they also “managed” several k8s clusters using… wait for it… AWS. I’d say, when you are all in AWS, then fine don’t use k8s. But you also don’t need Nomad then in most cases. And if you do need nomad on AWS then that’s fine as well, but it’s not comparable to k8s in general.
- OJFord 3y ago
  I think that was confusingly written, but what it means is 'k8s recommends not using it for database, but in Nomad is fine' - the EC2 instances are just nodes like any other, but 'tagged for db'. And if self-hosting Nomad on AWS is (allegedly) so much easier than operating managed k8s (EKS), isn't that more damning rather than less? The all-in AWS answer I suppose would be ECS. I quite like it but I think for anything beyond small, more than a few services part of the same system, I can understand wanting to run Nomad (or use EKS) instead.
  - GauntletWizard 3y ago
    You can run databases that way in K8s as well. Nobody does, because it's a terrible abstraction for high availability of conventional databases. The operator pattern does the opposite of help, creating more moving parts. The problem with running databases yourself is that it's hard to run databases, and having dedicated physical hardware works, until it doesn't. Nomad make it easy to run dedicated physical hardware for it. As soon as you have a failed EBS volume with your DB on it. Failed EBS volumes still happen frequently, though we've mostly obviated the problem by using managed dbs.
    - Semenoto 3y ago
      I'm running plenty of PostgreSQL in k8s. Know a few people who run hundreds of them too. And it's perfect for dev shit were colleges pull in their PostgreSQL themselves until later through helm dependencies
      - GauntletWizard 3y ago
        Oh, yeah - If you don't care about data availability/uptime, databases in k8s can be magical. It's just not suitable for high availability out of the box - Postgres, not k8s.
    - OJFord 3y ago
      I'm not defending the position, I've never used Nomad (though been wanting to, at home) but have used k8s, including in-cluster database. I was just explaining what the article was saying, because I think 'Nomad, plus EC2 for the db' is a misreading.
ranting-moth 3y ago
To whoever wrote this, I have this unsolicited advice: being obnoxious about your competitor is very off-putting to the reader. Putting down your competitor is a very fine art if you don't want to come across as a douche. Done correctly it can be hilarious though. But this is just a bull in the china shop.
- capableweb 3y ago
  > being obnoxious about your competitor As far as I can tell, the author of the article doesn't work for Hashicorp (Nomad) nor Google (Kubernetes) so not sure why you're giving this unsolicited advice?
  - ranting-moth 3y ago
    That's OK.
edude03 3y ago
I’ve always hated this style of argument in tech where someone says X can do what Y does as an argument against using Y (or for using X) because it always missed *why* people use Y in the first place. See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code) Or you can do machine learning in JavaScript so python is useless (you can but it’s a much smaller ecosystem) Etc etc. You can pick whatever tools you want for whatever reason but please give me a better reason than it *can* be done.
- worksonmine 3y ago
  > See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code) Isn't it usually the other way around? "This would be safer if written in rust". > Or you can do machine learning in JavaScript so python is useless Said no one ever. The author also elaborates why (according to him) X is preferable over Y (mainly simplicity), and even if he didn't there's value in comparing tools if one is on the fence of switching but afraid of loosing compatibility. Entire websites exist with that sole purpose.
  - edude03 3y ago
    >> Or you can do machine learning in JavaScript so python is useless > Said no one ever. I want to reply to this point because I have some relevant & interesting experience here. I've done a fair amount of consulting for companies who have a scenario that goes like this: - founder starts business in their free time, in a language they know well, as a monolith. - founder gets a few customers and decides to scale the business - founder decides they need to use "AI" to solve some business problem - founder hires a person who is great at solving that particular problem. - that person wants to use python (of course they do) but the monolith is not in python so various tensions ensue. - the founder (or someone else on the engineer team) inevitably says "you can do ML in $LANGUAGE_WE_CHOSE, why use python? We didn't chose python because $REASONS_PYTHON_SUCKS. You'd be surprised how often the language they chose is javascript and how often the reason not to use python is that they can't share code between the client and server. So unfortunately that example wasn't pulled out of nowhere
nunez 3y ago
I wish more places ran Nomad. I haven't used it myself but it seems so much simpler than Kubernetes (and, like the author, I have extensive experience operating Kubernetes clusters) and HCL is a dream compared to drowning in OpenAPI docs and Golang structs to figure out how the hell to define the MySuperCoolCR resource that x.y.x-k8s.example.com/v1beta4 offers.
- McSwag 3y ago
  I’ve been using it in production for many years and I agree that it’s vastly simpler than kubernetes, by many orders of magnitude. I’m currently migrating off of it because I reached a point where I needed more enterprise-y features for not a ton of money such as rbac, federation, multi-tenant isolation, resource quotas, and less-often mentioned, hireability (it’s easier to hire for since more people have had exposure to it). HashiCorps enterprise products are also notoriously expensive. Kubernetes has a lot of merits and open source community support which is attractive which means out of the box integrations are vast. That said, Nomad builds in a lot of sensible default behaviors that are way way closer to production ready with significantly less effort. Many things in Nomad, with Consul and Vault “just work” like you’d expect. Some of it seems like magic. Kubernetes on the other hand does almost nothing for you to make your custom app production ready out of the box. This means there’s a ton of room for error and incorrectly configured applications. CNIs, Ingress, PodAffinity, PodDisruptionBudgets, Services, HPAs, ServiceRoles, Secrets management are not defined for you and must be exactly correct or orchestrated in clunky and obscure ways, or a ton of effort needs to go into filling these gaps. “Just use EKS/GKE” quickly becomes a list of subsequent decision trees and you need to hope yoy made the right decision because if you don’t your choice will be deprecated in 6-9 months. The two config options for Kubernetes are Helm/Kustomize Yaml. Both are great for quick POCs but at production scale are completely inferior to HCL and generally an absolute nightmare to work with on the day-to-day. If I’m a team of one in a startup, I’d choose Nomad.

36 points by madmax108 3y ago

fesc 3y ago

I stopped reading when the solution for DNS was AWS and the solution for stateful sets was AWS EC2. This is just a very weird comparison. Seems like the author doesn’t understand why people run K8s. Of course they also “managed” several k8s clusters using… wait for it… AWS. I’d say, when you are all in AWS, then fine don’t use k8s. But you also don’t need Nomad then in most cases. And if you do need nomad on AWS then that’s fine as well, but it’s not comparable to k8s in general.

OJFord 3y ago
I think that was confusingly written, but what it means is 'k8s recommends not using it for database, but in Nomad is fine' - the EC2 instances are just nodes like any other, but 'tagged for db'. And if self-hosting Nomad on AWS is (allegedly) so much easier than operating managed k8s (EKS), isn't that more damning rather than less? The all-in AWS answer I suppose would be ECS. I quite like it but I think for anything beyond small, more than a few services part of the same system, I can understand wanting to run Nomad (or use EKS) instead.
- GauntletWizard 3y ago
  You can run databases that way in K8s as well. Nobody does, because it's a terrible abstraction for high availability of conventional databases. The operator pattern does the opposite of help, creating more moving parts. The problem with running databases yourself is that it's hard to run databases, and having dedicated physical hardware works, until it doesn't. Nomad make it easy to run dedicated physical hardware for it. As soon as you have a failed EBS volume with your DB on it. Failed EBS volumes still happen frequently, though we've mostly obviated the problem by using managed dbs.
  - Semenoto 3y ago
    I'm running plenty of PostgreSQL in k8s. Know a few people who run hundreds of them too. And it's perfect for dev shit were colleges pull in their PostgreSQL themselves until later through helm dependencies
    - GauntletWizard 3y ago
      Oh, yeah - If you don't care about data availability/uptime, databases in k8s can be magical. It's just not suitable for high availability out of the box - Postgres, not k8s.
  - OJFord 3y ago
    I'm not defending the position, I've never used Nomad (though been wanting to, at home) but have used k8s, including in-cluster database. I was just explaining what the article was saying, because I think 'Nomad, plus EC2 for the db' is a misreading.

ranting-moth 3y ago

To whoever wrote this, I have this unsolicited advice: being obnoxious about your competitor is very off-putting to the reader. Putting down your competitor is a very fine art if you don't want to come across as a douche. Done correctly it can be hilarious though. But this is just a bull in the china shop.

capableweb 3y ago
> being obnoxious about your competitor As far as I can tell, the author of the article doesn't work for Hashicorp (Nomad) nor Google (Kubernetes) so not sure why you're giving this unsolicited advice?
- ranting-moth 3y ago
  That's OK.

edude03 3y ago

I’ve always hated this style of argument in tech where someone says X can do what Y does as an argument against using Y (or for using X) because it always missed *why* people use Y in the first place. See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code) Or you can do machine learning in JavaScript so python is useless (you can but it’s a much smaller ecosystem) Etc etc. You can pick whatever tools you want for whatever reason but please give me a better reason than it *can* be done.

worksonmine 3y ago
> See for example you can write safe code in C therefore rust is pointless (yes but rust makes it much easier to write safe code) Isn't it usually the other way around? "This would be safer if written in rust". > Or you can do machine learning in JavaScript so python is useless Said no one ever. The author also elaborates why (according to him) X is preferable over Y (mainly simplicity), and even if he didn't there's value in comparing tools if one is on the fence of switching but afraid of loosing compatibility. Entire websites exist with that sole purpose.
- edude03 3y ago
  >> Or you can do machine learning in JavaScript so python is useless > Said no one ever. I want to reply to this point because I have some relevant & interesting experience here. I've done a fair amount of consulting for companies who have a scenario that goes like this: - founder starts business in their free time, in a language they know well, as a monolith. - founder gets a few customers and decides to scale the business - founder decides they need to use "AI" to solve some business problem - founder hires a person who is great at solving that particular problem. - that person wants to use python (of course they do) but the monolith is not in python so various tensions ensue. - the founder (or someone else on the engineer team) inevitably says "you can do ML in $LANGUAGE_WE_CHOSE, why use python? We didn't chose python because $REASONS_PYTHON_SUCKS. You'd be surprised how often the language they chose is javascript and how often the reason not to use python is that they can't share code between the client and server. So unfortunately that example wasn't pulled out of nowhere

nunez 3y ago

I wish more places ran Nomad. I haven't used it myself but it seems so much simpler than Kubernetes (and, like the author, I have extensive experience operating Kubernetes clusters) and HCL is a dream compared to drowning in OpenAPI docs and Golang structs to figure out how the hell to define the MySuperCoolCR resource that x.y.x-k8s.example.com/v1beta4 offers.

McSwag 3y ago
I’ve been using it in production for many years and I agree that it’s vastly simpler than kubernetes, by many orders of magnitude. I’m currently migrating off of it because I reached a point where I needed more enterprise-y features for not a ton of money such as rbac, federation, multi-tenant isolation, resource quotas, and less-often mentioned, hireability (it’s easier to hire for since more people have had exposure to it). HashiCorps enterprise products are also notoriously expensive. Kubernetes has a lot of merits and open source community support which is attractive which means out of the box integrations are vast. That said, Nomad builds in a lot of sensible default behaviors that are way way closer to production ready with significantly less effort. Many things in Nomad, with Consul and Vault “just work” like you’d expect. Some of it seems like magic. Kubernetes on the other hand does almost nothing for you to make your custom app production ready out of the box. This means there’s a ton of room for error and incorrectly configured applications. CNIs, Ingress, PodAffinity, PodDisruptionBudgets, Services, HPAs, ServiceRoles, Secrets management are not defined for you and must be exactly correct or orchestrated in clunky and obscure ways, or a ton of effort needs to go into filling these gaps. “Just use EKS/GKE” quickly becomes a list of subsequent decision trees and you need to hope yoy made the right decision because if you don’t your choice will be deprecated in 6-9 months. The two config options for Kubernetes are Helm/Kustomize Yaml. Both are great for quick POCs but at production scale are completely inferior to HCL and generally an absolute nightmare to work with on the day-to-day. If I’m a team of one in a startup, I’d choose Nomad.