3 ms·
For work I can totally understand this trade-off but this fact makes k8s pretty much a non-starter on side projects or personal sites because running a small cl
by Spivak 5y ago
For work I can totally understand this trade-off but this fact makes k8s pretty much a non-starter on side projects or personal sites because running a small cluster on small nodes leaves you so little headroom for your code that it's just not worth it. Swarm/Nomad/pacemaker all sip resources by comparison.
- egberts1 5y ago^this^
- hinkley 5y agoI got all set up with a Pi-clone cluster and then K3s decided that an extra 100MB of memory savings was not worth maintaining a separate implementation of part of the services. On a 16GB machine that's not a big deal. On a 2GB machine that's a significant fraction of your disk cache, for a very slow disk. On a 1GB machine that's just make or break. ETA: And you may think that these toys don't matter, but people have to learn and buy-in somewhere, and the fact of the matter is that I have production services right now that mostly need bandwidth, not CPU or memory, and once I tune them to the most appropriate AWS instance for that workload, the Kubernetes overhead would become the plurality of resource usage on these boxes (if I were using k8s, which we are not at present). K8s only scales by getting into bin packing, where this latency-sensitive service runs with substantially more potentially noisy neighbors.
- hobofan 5y agoI mean for GKE, there is a free tier that covers the cost of a autopilot cluster[0], which means basically no overhead cost of running on GKE instead of a Compute Engine node. [0]: https://cloud.google.com/kubernetes-engine/pricing#cluster_management_fee_and_free_tier https://cloud.google.com/kubernetes-engine/pricing#cluster_m...
- p_l 5y agoThe history of that is actually a bit funny. You see, there used to be no such "fixed overhead" in early versions (including post 1.0) of k8s. This turned out to be a worse idea for less involved or experienced operators than fixed overhead, because it turned out that people would run k8s nodes way underpowered, load them with a ton of workload, then have lots of outages as they starved critical system components of resources. Because of that a (settable) "overhead minimum" was added to calculation done about available resources, iirc originally going for 0.9 cpu core and I don't recall how much memory. This allowed to still run a 2 core node (though it's really not recommended) for experimentation, while greatly lowering chances of obscure for newbies issues. It didn't prevent them (PSA: If another team provisions your cluster, check what resources they provisioned...) but it makes it much harder to fail.
- sandGorgon 5y agoactually that is not really true - i strongly urge you to try out http://k3s.io/ http://k3s.io/ or https://k0sproject.io/ https://k0sproject.io/ these are full-fledged, certified k8s distributions that run on raspberry pi as well as all the way in production. https://www.youtube.com/results?search_query=raspberry+pi+k3s https://www.youtube.com/results?search_query=raspberry+pi+k3...