3 ms·

k8s doesn't get you scaling, in fact, it can get in the way, but if you play your cards right, it gets you high utilization of the CPU and RAM for which you pay

k8s doesn't get you scaling, in fact, it can get in the way, but if you play your cards right, it gets you high utilization of the CPU and RAM for which you pay, more than doing something simpler via autoscaling or whatever, because it's a solution to dynamic workload distribution, which is a difficult thing to build yourself.

dinvlad 4y ago
Oh, after trying my hand at a custom scheduler on bare-bones VMs, I dreamed of using K8s for bin-packing too, but in reality they still end up just running a bunch of Java services that are extremely poor at RAM reporting, and so really are not compatible with the sexy VPA and other such cool stuff. So the end result is still just a bunch of nodes running with way too much "free" RAM (and hence CPU) to accommodate for poorly predictable JRE RAM consumption patterns. I once observed that what could run on a single node with 16 cores (according to GKE's own cost reporting metrics) ends up running on a cluster of 512 (!) vCPUs. So it's more than a 10x waste, just purely to avoid running OOM etc.
- edmcnulty101 4y ago
  Is that k8's that takes up to much RAM?
  - oppositelock 4y ago
    No, the k8s "kubelet", which runs on each worker node, doesn't use much RAM. Java services have notoriously spiky memory usage, so you end up provisioning much more RAM than you need in the average case to be able to support the spikes, so you underutilize RAM in the average case. The previous poster is describing an issue where they overprovisioned RAM heavily, and so, ended up using a lot of nodes due to the way memory requests and limits are managed in k8s. You fix this by enabling swap and and allocating pods to nodes based on their common memory usage, and accept that your worker node will slow down when some Java process wants all the RAM.
    - dinvlad 4y ago
      Tbh, k8s "system" namespaces also consume quite a bit (particulary if you wanna run a minimal system) - at least 0.5 vCPU on each node and something like 0.5-1GB of RAM. This is only important for the smallest systems, but still is a hindrance to K8s adoption for such projects.

dinvlad 4y ago

Oh, after trying my hand at a custom scheduler on bare-bones VMs, I dreamed of using K8s for bin-packing too, but in reality they still end up just running a bunch of Java services that are extremely poor at RAM reporting, and so really are not compatible with the sexy VPA and other such cool stuff. So the end result is still just a bunch of nodes running with way too much "free" RAM (and hence CPU) to accommodate for poorly predictable JRE RAM consumption patterns. I once observed that what could run on a single node with 16 cores (according to GKE's own cost reporting metrics) ends up running on a cluster of 512 (!) vCPUs. So it's more than a 10x waste, just purely to avoid running OOM etc.

edmcnulty101 4y ago
Is that k8's that takes up to much RAM?
- oppositelock 4y ago
  No, the k8s "kubelet", which runs on each worker node, doesn't use much RAM. Java services have notoriously spiky memory usage, so you end up provisioning much more RAM than you need in the average case to be able to support the spikes, so you underutilize RAM in the average case. The previous poster is describing an issue where they overprovisioned RAM heavily, and so, ended up using a lot of nodes due to the way memory requests and limits are managed in k8s. You fix this by enabling swap and and allocating pods to nodes based on their common memory usage, and accept that your worker node will slow down when some Java process wants all the RAM.
  - dinvlad 4y ago
    Tbh, k8s "system" namespaces also consume quite a bit (particulary if you wanna run a minimal system) - at least 0.5 vCPU on each node and something like 0.5-1GB of RAM. This is only important for the smallest systems, but still is a hindrance to K8s adoption for such projects.