4 ms·

I do believe that with the right knowledge of Kubernetes internals it's probably possible to get k8s cold start times competitive with where we landed without K

by paulgb 3y ago

I do believe that with the right knowledge of Kubernetes internals it's probably possible to get k8s cold start times competitive with where we landed without Kubernetes (generally subsecond, often under 0.5s depending on how much the container does before passing a health check), but we'd have to understand k8s internals really well and would have ended up throwing out much of what already existed. And we'd probably end up breaking most of the reasons for using Kubernetes in the first place in the process.

p_l 3y ago
Not much internals needed, but actual in depth understanding of Pod kube-api plus at least basics of how scheduler, kubelet, and kubelet drivers interact. Big possible win is custom scheduling, but barely anyone seems to know it exists
- paulgb 3y ago
  Yeah, looking into writing a scheduler was basically where we stepped back and said “if we write this ourselves, why not the rest, too”. As I see it, the biggest gains that we were able to get were by making things happen in parallel that would by default happen in sequence, and optimizing for the happy path instead of optimizing for reducing failure. In Kubernetes it's reasonable to have to wait for a dozen things to serially go through RAFT consensus in etcd before the pod runs, but we don't want that. (I made up the dozen number, but my point is that that design would be perfectly acceptable given Kubernetes' design constraints)
- cogman10 3y ago
  Not surprising to me. People are complaining about how difficult it is to know k8s when you talk about the basic default objects. Getting into the weeds of how the api and control plane work (especially since it has little impact on day to day dev) is something devs tend to just avoid.
  - p_l 3y ago
    Honestly, devs of the applications that run on top probably should not have to worry about it. Instead have a platform team provide the necessary features.
hobofan 3y ago
Yeah, with plain Kubernetes I'd also see the practical limit around ~0.5s. If you are on GKE Autopilot where you also have little control over node startup there is likely also a lot more unpredictability. Something like Knative can allow for faster startup times if you follow the common best-practices (pre-fetching images, etc.), but I'm not sure if it supports enough of the session-related feature that you were probably looking for to be a stand-in for Plane.

by paulgb 3y ago

p_l 3y ago

Not much internals needed, but actual in depth understanding of Pod kube-api plus at least basics of how scheduler, kubelet, and kubelet drivers interact. Big possible win is custom scheduling, but barely anyone seems to know it exists

paulgb 3y ago
Yeah, looking into writing a scheduler was basically where we stepped back and said “if we write this ourselves, why not the rest, too”. As I see it, the biggest gains that we were able to get were by making things happen in parallel that would by default happen in sequence, and optimizing for the happy path instead of optimizing for reducing failure. In Kubernetes it's reasonable to have to wait for a dozen things to serially go through RAFT consensus in etcd before the pod runs, but we don't want that. (I made up the dozen number, but my point is that that design would be perfectly acceptable given Kubernetes' design constraints)
cogman10 3y ago
Not surprising to me. People are complaining about how difficult it is to know k8s when you talk about the basic default objects. Getting into the weeds of how the api and control plane work (especially since it has little impact on day to day dev) is something devs tend to just avoid.
- p_l 3y ago
  Honestly, devs of the applications that run on top probably should not have to worry about it. Instead have a platform team provide the necessary features.

hobofan 3y ago

Yeah, with plain Kubernetes I'd also see the practical limit around ~0.5s. If you are on GKE Autopilot where you also have little control over node startup there is likely also a lot more unpredictability. Something like Knative can allow for faster startup times if you follow the common best-practices (pre-fetching images, etc.), but I'm not sure if it supports enough of the session-related feature that you were probably looking for to be a stand-in for Plane.