7 ms·

Good points in here. The one thing about Kubernetes that I think teams need to be wary of is that upgrading needs to be treated as a first class citizen. In ord

by cfors 7y ago

Good points in here. The one thing about Kubernetes that I think teams need to be wary of is that upgrading needs to be treated as a first class citizen. In order to use the awesome tooling that so many people are building in Kubernetes (see basically all of the CNCF projects), the cluster API needs to be kept up to date. Once you start falling behind, you run the risks of being stuck in a version compatibility nightmare. Other than that, I can't imagine not running non-stateful applications on something other than Kubernetes anymore.

shuss 7y ago
The one point I keep driving is Kubernetes does so much useful stuff that is a pain to manage otherwise, that it's totally worth the risk currently posed by its rough edges. I'm hoping that came out clear in the article.
- lmeyerov 7y ago
  Maybe this is the analogy: k8s is now the infra equiv of npm / js frontends, where you need to run to stay in place, and will increase day-to-day strain on everyone. If you're ok with that price b/c your other things are so on fire in a few very specific dev+services at-scale ways, start. Otherwise, give it another X years before you slow down your team(s) & business.
SPascareli13 7y ago
> Other than that, I can't imagine not running non-stateful applications on something other than Kubernetes anymore. How do you deal with stateful applications like databases?
- havetocharge 7y ago
  You need to break up your app into stateful and stateless parts, run stateless ones using Kubernetes, and continue running the stateful ones the way you do now.
- nailer 7y ago
  I'm not into containers (in much the same way the article embraces 'herds not pets' we serverless folks embrace 'cockroaches not herds') but you can connect to DBs from a lambda in <20 ms. The DB itself is typically provided by the cloud provider and is usually a traditional cluster setup. Is it stateful? Yes, but you could throw it into a river, not know anything about how it was configured, connect another box to the cluster and get the capacity back. Exactly like whatever was running the lambda.
- markbnj 7y ago
  > How do you deal with stateful applications like databases? The impedance issues between kubernetes and stateful applications have been mostly solved. The main problems stemmed from the object model not yet being mature enough. Deployments did not offer good solutions for assigning stable identities to pods across restarts, or making it possible for them to find and attach to persistent volumes. StatefulSets and PersistentVolumeClaims solve these problems. Most of our k8s workloads are stateless, but we do run some big elasticsearch workloads in it. Having said that, I think if you're a small- to mid-sized business running on the cloud managed offerings are the first place to look. GCP CloudSQL has worked extremely well for us.
  - SPascareli13 7y ago
    What about a on prem kubernetes cluster, would you use kubernetes for something like a MySQL database, or run it outside or the container environment and just link to it from the stateless services in the cluster?
    - majewsky 7y ago
      Not OP, but my team is running on 100% k8s, mostly on bare metal. We do everything on k8s because of principle of least surprise: When apps are running as Deployments or StatefulSets, everyone knows how to deal with them (how to find logs, how to restart etc.) and it fits in the established deployment procedures (everything is a Helm chart etc.). When you have a single stateful service with a completely different configuration (e.g. RDBMS on a dedicated machine), it's always going to be "the odd-ball that only Greg knows how to upgrade". The whole point of k8s is not having these odd-balls. We have all databases in k8s, with physical volumes backed by an NFS-connected storage that's sitting in the same rack.
      - houseofzeus 7y ago
        Do you find you are getting a dense enough # of pods per host on bare-metal?
        7y ago

by cfors 7y ago

shuss 7y ago

The one point I keep driving is Kubernetes does so much useful stuff that is a pain to manage otherwise, that it's totally worth the risk currently posed by its rough edges. I'm hoping that came out clear in the article.

lmeyerov 7y ago
Maybe this is the analogy: k8s is now the infra equiv of npm / js frontends, where you need to run to stay in place, and will increase day-to-day strain on everyone. If you're ok with that price b/c your other things are so on fire in a few very specific dev+services at-scale ways, start. Otherwise, give it another X years before you slow down your team(s) & business.

SPascareli13 7y ago

> Other than that, I can't imagine not running non-stateful applications on something other than Kubernetes anymore. How do you deal with stateful applications like databases?

havetocharge 7y ago
You need to break up your app into stateful and stateless parts, run stateless ones using Kubernetes, and continue running the stateful ones the way you do now.
nailer 7y ago
I'm not into containers (in much the same way the article embraces 'herds not pets' we serverless folks embrace 'cockroaches not herds') but you can connect to DBs from a lambda in <20 ms. The DB itself is typically provided by the cloud provider and is usually a traditional cluster setup. Is it stateful? Yes, but you could throw it into a river, not know anything about how it was configured, connect another box to the cluster and get the capacity back. Exactly like whatever was running the lambda.
markbnj 7y ago
> How do you deal with stateful applications like databases? The impedance issues between kubernetes and stateful applications have been mostly solved. The main problems stemmed from the object model not yet being mature enough. Deployments did not offer good solutions for assigning stable identities to pods across restarts, or making it possible for them to find and attach to persistent volumes. StatefulSets and PersistentVolumeClaims solve these problems. Most of our k8s workloads are stateless, but we do run some big elasticsearch workloads in it. Having said that, I think if you're a small- to mid-sized business running on the cloud managed offerings are the first place to look. GCP CloudSQL has worked extremely well for us.
- SPascareli13 7y ago
  What about a on prem kubernetes cluster, would you use kubernetes for something like a MySQL database, or run it outside or the container environment and just link to it from the stateless services in the cluster?
  - majewsky 7y ago
    Not OP, but my team is running on 100% k8s, mostly on bare metal. We do everything on k8s because of principle of least surprise: When apps are running as Deployments or StatefulSets, everyone knows how to deal with them (how to find logs, how to restart etc.) and it fits in the established deployment procedures (everything is a Helm chart etc.). When you have a single stateful service with a completely different configuration (e.g. RDBMS on a dedicated machine), it's always going to be "the odd-ball that only Greg knows how to upgrade". The whole point of k8s is not having these odd-balls. We have all databases in k8s, with physical volumes backed by an NFS-connected storage that's sitting in the same rack.
    - houseofzeus 7y ago
      Do you find you are getting a dense enough # of pods per host on bare-metal?
      - 7y ago