4 ms·

How's k3s doing these days? I'm not familiar with the area, but it was marketing itself as a simpler k8s.

theossuary 3y ago
K3s is fine, but it was started on an older version of kubernetes, so there's a few places where K8s has solved an issue (like bootstrapping) that k3s had to solve already in a different way. K0s is a new project I like that is similar to k3s in spirit. Microk8s is a snap version from canonical with some neat out of the box stuff (like the simplest GPU support); but again, only available as a snap. Honestly though just use kubespray for production, it's fine
- gchamonlive 3y ago
  Kubespray is fine if you can spare hours of Dev time. Where I used to work I wasn't involved directly with maintaining the k8s cluster using kubespray, but from the daily meetings it was clear that it is no easy task. Sure it helps with deploying a serviceable production cluster, but when you start throwing at it things like disaster recovery, the automation scripts can grow quite large and complex, even leveraging kubespray. If you can, use a managed service like gks.
  - quickthrower2 3y ago
    I was going to say. Using a managed service is specialized enough. I can't imagine building one up with everything from scratch. Feels like a full time role "Kubernetes Sys Admin"
- cyberpunk 3y ago
  kubespray is not good advice unless it's changed massively since I last used it in 2020 or so. Imagine a 3 hour Ansible-playbook execution, which randomly failed on different tasks, most of which got fixed by hitting <up><enter> until it applied... We switched to RKE, which runs even on our bigger clusters (nothing mad, maybe 40 nodes) in a few minutes and works perfectly..
  - antod 3y ago
    Yeah kubespray was awful. Random results, no idempotency, slow, fragile, hard to upgrade k8s. We had much better luck just scripting kubeadm over ssh orchestration tooling.
- doctorpangloss 3y ago
  microk8s currently has a showstopping issue that makes it guaranteed to have an irrecoverable failure in HA mode. see https://github.com/canonical/microk8s/issues/3227 https://github.com/canonical/microk8s/issues/3227 k0s is better but also has a lot of bugs. it's the closest to vanilla kubernetes among all the distributions. > like the simplest GPU support linux users should be ready to install the nvidia device plugin. if they can't do that, they're never going to succeed in running a gpu accelerated application on their cluster anyway. > like bootstrapping in my experience, writing all the bootstrap scripts is painful. but now that there's chatgpt, so much of the drudgery as gone away.
husarcik 3y ago
K3s is the best out of any of these "lite" k8 frameworks. As someone who has never worked in tech, it took me about five hours to setup k3s with cert manager with a few PHP and nodejs deployments. This is using my own helm setup making it trivial to upgrade.

K3s is fine, but it was started on an older version of kubernetes, so there's a few places where K8s has solved an issue (like bootstrapping) that k3s had to solve already in a different way. K0s is a new project I like that is similar to k3s in spirit. Microk8s is a snap version from canonical with some neat out of the box stuff (like the simplest GPU support); but again, only available as a snap. Honestly though just use kubespray for production, it's fine

gchamonlive 3y ago
Kubespray is fine if you can spare hours of Dev time. Where I used to work I wasn't involved directly with maintaining the k8s cluster using kubespray, but from the daily meetings it was clear that it is no easy task. Sure it helps with deploying a serviceable production cluster, but when you start throwing at it things like disaster recovery, the automation scripts can grow quite large and complex, even leveraging kubespray. If you can, use a managed service like gks.
- quickthrower2 3y ago
  I was going to say. Using a managed service is specialized enough. I can't imagine building one up with everything from scratch. Feels like a full time role "Kubernetes Sys Admin"
cyberpunk 3y ago
kubespray is not good advice unless it's changed massively since I last used it in 2020 or so. Imagine a 3 hour Ansible-playbook execution, which randomly failed on different tasks, most of which got fixed by hitting <up><enter> until it applied... We switched to RKE, which runs even on our bigger clusters (nothing mad, maybe 40 nodes) in a few minutes and works perfectly..
- antod 3y ago
  Yeah kubespray was awful. Random results, no idempotency, slow, fragile, hard to upgrade k8s. We had much better luck just scripting kubeadm over ssh orchestration tooling.
doctorpangloss 3y ago
microk8s currently has a showstopping issue that makes it guaranteed to have an irrecoverable failure in HA mode. see https://github.com/canonical/microk8s/issues/3227 https://github.com/canonical/microk8s/issues/3227 k0s is better but also has a lot of bugs. it's the closest to vanilla kubernetes among all the distributions. > like the simplest GPU support linux users should be ready to install the nvidia device plugin. if they can't do that, they're never going to succeed in running a gpu accelerated application on their cluster anyway. > like bootstrapping in my experience, writing all the bootstrap scripts is painful. but now that there's chatgpt, so much of the drudgery as gone away.

husarcik 3y ago

K3s is the best out of any of these "lite" k8 frameworks. As someone who has never worked in tech, it took me about five hours to setup k3s with cert manager with a few PHP and nodejs deployments. This is using my own helm setup making it trivial to upgrade.