3 ms·

We've been doing this because alternatives like k8s burned out time, $, and other team members, and our on-prem customers like it operationally vs k8s. SW teams

We've been doing this because alternatives like k8s burned out time, $, and other team members, and our on-prem customers like it operationally vs k8s. SW teams ran high uptime stuff before k8s, and you can still do most of those tricks. We're revisiting, but selectively and only b/c we're now getting into trickier scenarios like multi-az and potentially elastic GPUs for our SaaS offering Ex: - blue/green stateless nodes - DNS/load balancer pointer flips - Do the DB as a managed saas in the cloud and only ^^^ for app servers - Backups: We've been simply syncing (rsync, pg_dump, ..) w/ basic backup testing as part of our launch flow (stressed as part of blue/green flow), and cloud services automate blob backups. We're moving to DBaaS for doing that better, which involves taking the DB out of our app for our SaaS users In practice, our service availability has been largely degraded by external infra factors, like Azure's DNS going down, than by not using k8s/swarms for internal infra. Likewise, our own logic bugs causing QoS issues that don't show up in our 9's reports.

rualca 5y ago
> We've been doing this because alternatives like k8s burned out time, $, and other team members, and our on-prem customers like it operationally vs k8s. It doesn't seem you bothered looking into alternatives. Take for example docker swarm. It already uses docker compose files (well, there are some differences) , they support blue-green deployments of stacks, and they even support multi-node deployments and horizontal scaling.
- lmeyerov 5y ago
  Swarm was/is a bad bet due to looking like Docker gave up on it, esp. for deploying to our on-prem users where we can't just change our mind the next day, and wanting something we'd have some faith in for 2-3 years. Nomad was more appealing from the simplicity view, esp. as we'd need to do some rewriting either way.

rualca 5y ago

> We've been doing this because alternatives like k8s burned out time, $, and other team members, and our on-prem customers like it operationally vs k8s. It doesn't seem you bothered looking into alternatives. Take for example docker swarm. It already uses docker compose files (well, there are some differences) , they support blue-green deployments of stacks, and they even support multi-node deployments and horizontal scaling.

lmeyerov 5y ago
Swarm was/is a bad bet due to looking like Docker gave up on it, esp. for deploying to our on-prem users where we can't just change our mind the next day, and wanting something we'd have some faith in for 2-3 years. Nomad was more appealing from the simplicity view, esp. as we'd need to do some rewriting either way.