3 ms·
We waited on this issue for 2 years before migrating off EKS Fargate to Managed Node Groups due to this very reason. In the end, it turned out to be much better
by booleanbetrayal 2y ago
We waited on this issue for 2 years before migrating off EKS Fargate to Managed Node Groups due to this very reason. In the end, it turned out to be much better of an environment anyway, because you didn't have to deal with Kubernetes oddities (like lack of Daemonsets) requiring anti-patterns and ultimately resulted in cheaper pricing due to better bin-packing. If you're doing K8s orchestration with Fargate, I highly recommend the switch.
- suryao 2y ago(founder of WarpBuild - we offer hosted GHA runners) This is a common issue. A common pain point we see with users approaching some scale with self-hosting on k8s is that the k8s node autoscaling can become inefficient because of spiky loads. We have a lot of users migrating off self-hosted setups using `actions-runner-controller` to ours because of this. Essentially, not having to deal with bin-packing is more efficient and concurrency, uptime guarantees are nice.
- watermelon0 2y agoTo be honest, lack of daemonsets makes sense, because you don't have hosts per se. Each pod is running on it's own Linux VM. Daemonsets are generally intended if you have multi-pod nodes; otherwise, you can just use sidecars.
- booleanbetrayal 2y agoSidecars are definitely a workaround but are hard to manage lifecycle for in conjunction with the primary container. This is now easier to do in 1.29+ with Sidecars officially supported via restartPolicy, but it was a colossal hassle prior to the advent of that. Also, we had noticed (maybe this has changed) that often logging frameworks were distro'd as DaemonSets (Helm, etc) and you'd have to shoe-horn the sidecar approach a bit.