4 ms·

> Other than that, I can't imagine not running non-stateful applications on something other than Kubernetes anymore. How do you deal with stateful applications

> Other than that, I can't imagine not running non-stateful applications on something other than Kubernetes anymore. How do you deal with stateful applications like databases?

havetocharge 7y ago
You need to break up your app into stateful and stateless parts, run stateless ones using Kubernetes, and continue running the stateful ones the way you do now.
nailer 7y ago
I'm not into containers (in much the same way the article embraces 'herds not pets' we serverless folks embrace 'cockroaches not herds') but you can connect to DBs from a lambda in <20 ms. The DB itself is typically provided by the cloud provider and is usually a traditional cluster setup. Is it stateful? Yes, but you could throw it into a river, not know anything about how it was configured, connect another box to the cluster and get the capacity back. Exactly like whatever was running the lambda.
markbnj 7y ago
> How do you deal with stateful applications like databases? The impedance issues between kubernetes and stateful applications have been mostly solved. The main problems stemmed from the object model not yet being mature enough. Deployments did not offer good solutions for assigning stable identities to pods across restarts, or making it possible for them to find and attach to persistent volumes. StatefulSets and PersistentVolumeClaims solve these problems. Most of our k8s workloads are stateless, but we do run some big elasticsearch workloads in it. Having said that, I think if you're a small- to mid-sized business running on the cloud managed offerings are the first place to look. GCP CloudSQL has worked extremely well for us.
- SPascareli13 7y ago
  What about a on prem kubernetes cluster, would you use kubernetes for something like a MySQL database, or run it outside or the container environment and just link to it from the stateless services in the cluster?
  - majewsky 7y ago
    Not OP, but my team is running on 100% k8s, mostly on bare metal. We do everything on k8s because of principle of least surprise: When apps are running as Deployments or StatefulSets, everyone knows how to deal with them (how to find logs, how to restart etc.) and it fits in the established deployment procedures (everything is a Helm chart etc.). When you have a single stateful service with a completely different configuration (e.g. RDBMS on a dedicated machine), it's always going to be "the odd-ball that only Greg knows how to upgrade". The whole point of k8s is not having these odd-balls. We have all databases in k8s, with physical volumes backed by an NFS-connected storage that's sitting in the same rack.
    - houseofzeus 7y ago
      Do you find you are getting a dense enough # of pods per host on bare-metal?
      - markbnj 7y ago
        That's an interesting question. Why would you anticipate a possible difference? Is the cluster running a different scheduler? If anything I would expect cloud-hosted managed clusters to possibly have less free resources due to running the pieces of the cloud provider's control plane.
        houseofzeus 7y ago
        There are bottlenecks in the Kubernetes architecture that start popping up once you cross about 250-300 Pods per node (the published limit in the K8S docs is actually 100). This is typically more of an issue when running the cluster directly on large bare-metal nodes because in the cloud or on virtualization the host gets split into more smaller nodes from a K8S perspective. Obviously this also depends on the characteristics of the workload, if you have a lot of monoliths that you have only barely containerized the Pod per Node limit may not be a concern at all even on bare-metal.
      - majewsky 7y ago
- jrochkind1 7y ago
  To someone who is not familiar with the technology beyond a high-level overview like in OP, the sentence "The main problems stemmed from the object model not yet being mature enough" sounds like "The main problems with stateful applications stemmed from there being problems with stateful applications".
  - markbnj 7y ago
    Fair enough. I could have said it more clearly. The strategies, api entities, and implementations to support stable pod identity and declarative persistent storage weren't there a couple of years ago, but now they are.
  - shaklee3 7y ago
    The problem was that these features came a little later after many of the stateless features were in there. Now we're talking about the stateful stuff being stable for many years, so all the arguments against it are usually old or uninformed.

> Other than that, I can't imagine not running non-stateful applications on something other than Kubernetes anymore. How do you deal with stateful applications like databases?

havetocharge 7y ago

You need to break up your app into stateful and stateless parts, run stateless ones using Kubernetes, and continue running the stateful ones the way you do now.

nailer 7y ago

I'm not into containers (in much the same way the article embraces 'herds not pets' we serverless folks embrace 'cockroaches not herds') but you can connect to DBs from a lambda in <20 ms. The DB itself is typically provided by the cloud provider and is usually a traditional cluster setup. Is it stateful? Yes, but you could throw it into a river, not know anything about how it was configured, connect another box to the cluster and get the capacity back. Exactly like whatever was running the lambda.

markbnj 7y ago

> How do you deal with stateful applications like databases? The impedance issues between kubernetes and stateful applications have been mostly solved. The main problems stemmed from the object model not yet being mature enough. Deployments did not offer good solutions for assigning stable identities to pods across restarts, or making it possible for them to find and attach to persistent volumes. StatefulSets and PersistentVolumeClaims solve these problems. Most of our k8s workloads are stateless, but we do run some big elasticsearch workloads in it. Having said that, I think if you're a small- to mid-sized business running on the cloud managed offerings are the first place to look. GCP CloudSQL has worked extremely well for us.

SPascareli13 7y ago
What about a on prem kubernetes cluster, would you use kubernetes for something like a MySQL database, or run it outside or the container environment and just link to it from the stateless services in the cluster?
- majewsky 7y ago
  Not OP, but my team is running on 100% k8s, mostly on bare metal. We do everything on k8s because of principle of least surprise: When apps are running as Deployments or StatefulSets, everyone knows how to deal with them (how to find logs, how to restart etc.) and it fits in the established deployment procedures (everything is a Helm chart etc.). When you have a single stateful service with a completely different configuration (e.g. RDBMS on a dedicated machine), it's always going to be "the odd-ball that only Greg knows how to upgrade". The whole point of k8s is not having these odd-balls. We have all databases in k8s, with physical volumes backed by an NFS-connected storage that's sitting in the same rack.
  - houseofzeus 7y ago
    Do you find you are getting a dense enough # of pods per host on bare-metal?
    - markbnj 7y ago
      That's an interesting question. Why would you anticipate a possible difference? Is the cluster running a different scheduler? If anything I would expect cloud-hosted managed clusters to possibly have less free resources due to running the pieces of the cloud provider's control plane.
      - houseofzeus 7y ago
        There are bottlenecks in the Kubernetes architecture that start popping up once you cross about 250-300 Pods per node (the published limit in the K8S docs is actually 100). This is typically more of an issue when running the cluster directly on large bare-metal nodes because in the cloud or on virtualization the host gets split into more smaller nodes from a K8S perspective. Obviously this also depends on the characteristics of the workload, if you have a lot of monoliths that you have only barely containerized the Pod per Node limit may not be a concern at all even on bare-metal.
    - majewsky 7y ago
jrochkind1 7y ago
To someone who is not familiar with the technology beyond a high-level overview like in OP, the sentence "The main problems stemmed from the object model not yet being mature enough" sounds like "The main problems with stateful applications stemmed from there being problems with stateful applications".
- markbnj 7y ago
  Fair enough. I could have said it more clearly. The strategies, api entities, and implementations to support stable pod identity and declarative persistent storage weren't there a couple of years ago, but now they are.
- shaklee3 7y ago
  The problem was that these features came a little later after many of the stateless features were in there. Now we're talking about the stateful stuff being stable for many years, so all the arguments against it are usually old or uninformed.