8 ms·

This article really resonated with me. We are starting to run into container orchestration problems but I really don’t like what I read about K8s. Apart from an

This article really resonated with me. We are starting to run into container orchestration problems but I really don’t like what I read about K8s. Apart from anything else, it seems designed for much bigger problems than mine, and require the kind of huge mental effort to understand which, ironically, will make it harder for my business to grow. I’m curious if you’ve taken a look at Nomad and the other HashiCorp tools? They appear focussed and compositional, as you say, and this is why we are probably going to adopt them instead of K8s - they seem to be in a strong position to replace the core of K8s with something simpler.

carlosf 5y ago
I use Nomad a lot in my company and I really like it. Our team tried to migrate to AWS ECS a few times and found it much harder to abstract stuff from devs / create self-service patterns. That said it's not a walk in the park. You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly.
- doctor_eval 5y ago
  Thanks. We're going to start small with just nomad, then vault, and as our needs grow we will probably adopt consul (we already use terraform so hopefully not a huge stretch) and maybe boundary. This is thing I like about the HashiCorp tools. You don't have to eat the whole cake in a single sitting.
- diragon 5y ago
  >You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly. I've been wondering, would it make sense to try to package all that into a single, hopefully simple and easily configurable, Linux image? And if it might be, why hasn't anyone done that yet?
- alephu5 5y ago
  There are some good ansible playbooks on GitHub for nomad, consul and vault. I personally don't use vault because it's overkill for the proeuct in working on at the moment. To avoid the pain of managing a CA and passing out certificates for TLS between services, I use a wireguard mesh and bind nomad, consul and vault to these wg interfaces. This includes all the chatter of these components, as well as the services I deploy with nomad. It's configured such that any job can join the "private" wireguard network or "public" internet gateway. It takes a few days to set up, but it's very easy to manage.
  - nsomaru 5y ago
    Do you have somewhere to point me to set things up in this configuration? I’m a freelancer that hosts client stuff and I need something between “SSH into server” and “kubernetes.” No, I never did buy the docker hype. Seem to be doing okay.
    - cpach 5y ago
      Have you looked into Fly.io or AWS Fargate?
deleted 5y ago
[deleted]
chubot 5y ago
I've only looked at the HashiCorp tools, not really used them. My understanding is they originated in a VM-based world (?), and I've worked almost exclusively with containers. I'm sure that has changed over time. I will say that I looked at HCL and it looks very nice: https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl But somehow it's not as popular as a mess of YAML and Go Templates? That genuinely leaves me scratching my head. I guess it's because people pick platforms and not languages? (BTW, in 2009 I designed and implemented the template language that Go templates are based on, and I find their common application pretty bizarre, e.g. in some Helm charts I looked at from this thread) Oil is growing a config dialect that looks a lot like HCL (although it's convergent evolution; I've never used it.) I think there is a lot of room for mixing declarative and imperative; as far as I can see HCL is mostly declarative (defining data structures). Anyway I'd be interested in reading about HashiCorp stuff but for some reason in my neck of the woods I don't hear too much about it. Maybe that's because they're paid services and the open source Kubernetes seems attractive by comparison? Or is it more of a VM vs. container thing?
- orangepenguin 5y ago
  All of the Hashicorp products are primarily open source products. While there are enterprise features and cloud-hosted versions of some of them, FOSS is the foundation of the company.

carlosf 5y ago

I use Nomad a lot in my company and I really like it. Our team tried to migrate to AWS ECS a few times and found it much harder to abstract stuff from devs / create self-service patterns. That said it's not a walk in the park. You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly.

doctor_eval 5y ago
Thanks. We're going to start small with just nomad, then vault, and as our needs grow we will probably adopt consul (we already use terraform so hopefully not a huge stretch) and maybe boundary. This is thing I like about the HashiCorp tools. You don't have to eat the whole cake in a single sitting.
diragon 5y ago
>You will need to scratch your head a little bit to setup consul + nomad + vault + a load balancer correctly. I've been wondering, would it make sense to try to package all that into a single, hopefully simple and easily configurable, Linux image? And if it might be, why hasn't anyone done that yet?
alephu5 5y ago
There are some good ansible playbooks on GitHub for nomad, consul and vault. I personally don't use vault because it's overkill for the proeuct in working on at the moment. To avoid the pain of managing a CA and passing out certificates for TLS between services, I use a wireguard mesh and bind nomad, consul and vault to these wg interfaces. This includes all the chatter of these components, as well as the services I deploy with nomad. It's configured such that any job can join the "private" wireguard network or "public" internet gateway. It takes a few days to set up, but it's very easy to manage.
- nsomaru 5y ago
  Do you have somewhere to point me to set things up in this configuration? I’m a freelancer that hosts client stuff and I need something between “SSH into server” and “kubernetes.” No, I never did buy the docker hype. Seem to be doing okay.
  - cpach 5y ago
    Have you looked into Fly.io or AWS Fargate?

deleted 5y ago

[deleted]

chubot 5y ago

I've only looked at the HashiCorp tools, not really used them. My understanding is they originated in a VM-based world (?), and I've worked almost exclusively with containers. I'm sure that has changed over time. I will say that I looked at HCL and it looks very nice: https://github.com/hashicorp/hcl https://github.com/hashicorp/hcl But somehow it's not as popular as a mess of YAML and Go Templates? That genuinely leaves me scratching my head. I guess it's because people pick platforms and not languages? (BTW, in 2009 I designed and implemented the template language that Go templates are based on, and I find their common application pretty bizarre, e.g. in some Helm charts I looked at from this thread) Oil is growing a config dialect that looks a lot like HCL (although it's convergent evolution; I've never used it.) I think there is a lot of room for mixing declarative and imperative; as far as I can see HCL is mostly declarative (defining data structures). Anyway I'd be interested in reading about HashiCorp stuff but for some reason in my neck of the woods I don't hear too much about it. Maybe that's because they're paid services and the open source Kubernetes seems attractive by comparison? Or is it more of a VM vs. container thing?

orangepenguin 5y ago
All of the Hashicorp products are primarily open source products. While there are enterprise features and cloud-hosted versions of some of them, FOSS is the foundation of the company.