5 ms·

> In some cases, it is too fast. When I installed K3s, all of the containers in the kube-system namespace kept entering the dreaded CrashLoopBackOff state [..]

by rkunde 4y ago

> In some cases, it is too fast. When I installed K3s, all of the containers in the kube-system namespace kept entering the dreaded CrashLoopBackOff state [..] After some investigation, I found out that the Mac Studio was just too fast for the Kubernetes resource timing I’d like to understand what the issue here is. Sounds counterintuitive to me.

adrianmsmith 4y ago
Classic that the only thing they reported didn't work was Kubernetes! Kubernetes is really such a complex and fragile system, at least in my experience.
linsomniac 4y ago
I read that as "the container ran so fast that the orchestration assumed it was failing".
taspeotis 4y ago
It’s called a race condition. They never expected something to complete before another thing.
- bornfreddy 4y ago
  Yup. It is also a bug and a very difficult one to catch, so original devs would probably be very grateful for report and help in fixing it. I hope OP opened an issue.
  - ofchnofc 4y ago
    [dead]
moondev 4y ago
Same here, OP can you share the limits you added? Or is it on namespace level
- jasoneckert 4y ago
  No problem. In the end, I had to actually set CPU limits on the pod level for traefik, svclb-traefik, metrics-server, coredns, and the local-path-provisioner. I initially chose a limit of "500m" for all of them except coredns (which seemed to be fine), but there were still some occasional issues with the metrics-server and traefik, so I increased those to "750m" and that solved those issues, but it caused coredns to CrashLoopBackOff. After setting a "500m" limit on coredns, everything ran smoothly. So, essentially, I set a "500m" on all of the pods in the kube-system namespace except for traefik and metrics-server (they got "750m"), and of course I didn't need to set limits on the helm-install-* pods. I didn't modify the default memory limits at all.

5 ms·

> In some cases, it is too fast. When I installed K3s, all of the containers in the kube-system namespace kept entering the dreaded CrashLoopBackOff state [..]

by rkunde 4y ago

adrianmsmith 4y ago
Classic that the only thing they reported didn't work was Kubernetes! Kubernetes is really such a complex and fragile system, at least in my experience.
linsomniac 4y ago
I read that as "the container ran so fast that the orchestration assumed it was failing".
taspeotis 4y ago
It’s called a race condition. They never expected something to complete before another thing.
- bornfreddy 4y ago
  Yup. It is also a bug and a very difficult one to catch, so original devs would probably be very grateful for report and help in fixing it. I hope OP opened an issue.
  - ofchnofc 4y ago
    [dead]
moondev 4y ago
Same here, OP can you share the limits you added? Or is it on namespace level
- jasoneckert 4y ago
  No problem. In the end, I had to actually set CPU limits on the pod level for traefik, svclb-traefik, metrics-server, coredns, and the local-path-provisioner. I initially chose a limit of "500m" for all of them except coredns (which seemed to be fine), but there were still some occasional issues with the metrics-server and traefik, so I increased those to "750m" and that solved those issues, but it caused coredns to CrashLoopBackOff. After setting a "500m" limit on coredns, everything ran smoothly. So, essentially, I set a "500m" on all of the pods in the kube-system namespace except for traefik and metrics-server (they got "750m"), and of course I didn't need to set limits on the helm-install-* pods. I didn't modify the default memory limits at all.