3 ms·

At a high level, your use-case maps well to a Kubernetes DaemonSet, which is a resource that represents the desire to run a copy of something containerized on e

At a high level, your use-case maps well to a Kubernetes DaemonSet, which is a resource that represents the desire to run a copy of something containerized on every node within your cluster. Kubernetes would give you some things that might be useful: * A unified control plane (monitoring, statusing) * An automated rolling-upgrade system * "self-healing" in the form of automatically restarting your containers if they go awry However, for your use-case, it is likely more effort than it's worth: * Settin up and maintaining Kubernetes on bare metal servers is a huge amount of work * A lot of Kubernetes features are not needed for your use-case * The Kubernetes networking model essentially requires you to set up private network spanning all of your sites in order to work but provides almost no benefit for your use-case

kissgyorgy 7y ago
Technically you could use a DaemonSet for that, but the primary use-case is not that for DaemonSets. It should be used for "daemon" containers, like logging daemons (e.g. fluentd), network plugin daemons like calico, flannel or monitoring like prometheus. If you want to run multiple instances of your application, you should define a replicaset or deployment with setting Pod affinity instead.
- clhodapp 7y ago
  That's super fiddly though: you will be leaning on (anti) affinity and counts to retain the 1:1 relationship between hosts and pods. I would argue that daemonsets aren't for any specific workload, they're more about how you want scheduling to happen (1 per node, on some subset of your nodes).
  - manigandham 7y ago
    This is where K8S becomes messy because of its rapid development history. Applications should just be a `Deployment` manifest with the type of distribution being a child setting (X replicas with affinity or 1-per-host). There's no reason to have competing ReplicaSets and DaemonSets just for this detail.
    - kissgyorgy 7y ago
      I disagree on this one, DaemonSets are for really a different use-case. There is no replicas field you can set, and strictly 1 Pod will run on 1 node. A Deployment is more flexible, because you can run even more Pods in different cities if necessary. What if there are more/less nodes are in different cities or different specs, which could and should run more Pods? You can't fine-tune DaemonSets for those cases, it really is for daemons.
      - manigandham 7y ago
        It's not about use-case, it's that "X number of replicas with affinity" OR "1-per-node" should just be a setting within a Deployment. No need to use a completely separate DaemonSet definition just to run 1-per-node.

kissgyorgy 7y ago

Technically you could use a DaemonSet for that, but the primary use-case is not that for DaemonSets. It should be used for "daemon" containers, like logging daemons (e.g. fluentd), network plugin daemons like calico, flannel or monitoring like prometheus. If you want to run multiple instances of your application, you should define a replicaset or deployment with setting Pod affinity instead.

clhodapp 7y ago
That's super fiddly though: you will be leaning on (anti) affinity and counts to retain the 1:1 relationship between hosts and pods. I would argue that daemonsets aren't for any specific workload, they're more about how you want scheduling to happen (1 per node, on some subset of your nodes).
- manigandham 7y ago
  This is where K8S becomes messy because of its rapid development history. Applications should just be a `Deployment` manifest with the type of distribution being a child setting (X replicas with affinity or 1-per-host). There's no reason to have competing ReplicaSets and DaemonSets just for this detail.
  - kissgyorgy 7y ago
    I disagree on this one, DaemonSets are for really a different use-case. There is no replicas field you can set, and strictly 1 Pod will run on 1 node. A Deployment is more flexible, because you can run even more Pods in different cities if necessary. What if there are more/less nodes are in different cities or different specs, which could and should run more Pods? You can't fine-tune DaemonSets for those cases, it really is for daemons.
    - manigandham 7y ago
      It's not about use-case, it's that "X number of replicas with affinity" OR "1-per-node" should just be a setting within a Deployment. No need to use a completely separate DaemonSet definition just to run 1-per-node.