5 ms·
we use nomad in production, no issues. Way less complex than k8s. We deploy Prometheus out of your list. We use NGINX, not Traefik. PG is deployed but not un
by zie 6y ago
we use nomad in production, no issues. Way less complex than k8s. We deploy Prometheus out of your list. We use NGINX, not Traefik. PG is deployed but not under Nomad, mostly just from laziness.
We run a local instance of send.firefox.com, redis, metabase, docker registry, a speedtest tool, a bunch of batch/periodic jobs, some internal custom services, all of our web hosting, etc directly out of nomad.
Our CI/CD runs make which builds the code, generates .nomad files and runs them, so it's super easy to re-deploy, rollback, etc.
k8s management on your own hardware is a full-time job. There are 2 of us that do all dev and ops for our product and there is 0% chance we could manage a k8s cluster w/ only 2 of us, but managing a nomad, consul and vault cluster is easy for us.
There are nomad templating tools, but we started using nomad back when it was brand new, so we just used sed for our meagre templating needs. Works a treat, and never saw any value to using some specialized tool(for our needs).
Totally recommend checking out nomad.
- WnZ39p0Dgydaz1 6y agoThanks for the insight, that sounds very promising! Right now I'm operating a k8s cluster on cloud VMs, so not managed, but not custom hardware either, myself and like you said it's pretty much a full-time job despite the relatively simple cluster workloads. It's great to hear from someone who successfully uses Nomad in production.
- aprdm 6y agoWhat is the day to day operation needs like ?
- miguendes 6y agoWhat about managed k8s solutions such as EKS? Would they also be a full time job? I understand that is probably more expensive but I'm curious if it would still be pain as having your own cluster.
- WnZ39p0Dgydaz1 6y agoThe main benefit you get from managed Kubernetes solutions is network/security, update- and scaling-related. These things are important, but they don't reduce the daily operating complexity around deployments (applications) on the cluster. In other words, they take the pain out of running and setting up Kubernetes itself, but they don't simplify using Kubernetes. I highly recommend using a managed solution unless you absolutely know what you are doing in terms of networking, security, and disaster recovery. They are not too expensive either, you're still mostly paying for the VM nodes. The only reason we're not using a managed solution is because we have some specific edge networking and multi-cluster requirements that are hard to do with a managed solution.
- zie 6y agoAgreed. And just to say, managing k8s on own hardware(or in cloud VM's) is a full-time position, which means you probably need more than 1 actual employee in that role, especially if you are a 24/7 shop, where you need 24/7 uptime and suddenly k8s is costing you a few full-time people just to keep k8s alive and healthy. That's a HUGE cost. I manage nomad on the side, I spend < 10% of my time babysitting nomad, consul and vault. Basically I just upgrade it occasionally. Otherwise I rarely think about it.
- deleted 6y ago[deleted]