9 ms·

Comparing K3s with vanilla Kubernetes

74 points by jhoelzel 4y ago

kobalsky 4y ago
the title doesn't make sense, it reads like "comparing Ubuntu with Linux, How Ubuntu is often the better choice". k3s is a kubernetes distribution
- ttymck 4y ago
  What is the official kubernetes distribution called?
  - iameli 4y ago
    There isn't one — the official docs give instructions for kubeadm, kOps, and kubespray. k3s would be an alternative to those.
    - ithkuil 4y ago
      Most of the other K8s distributions use the same K8s binaries. K3s is slightly different in that it's a trimmed down build of the same codebase
    - MuffinFlavored 4y ago
      i think you mean kubeadm and not kubeam
      - iameli 4y ago
        ty, edited
- jhoelzel 4y ago
  while true, writing "vanialla kubernetes" did not have the right feel for me. What would have been a better title for you?
  - user61129028 4y ago
    "vanilla kubernetes" isn't a distribution either. > Kubernetes and k3s are both container orchestration platforms Your article makes it seem like k3s and k8s are different platforms which is simply incorrect. A more useful comparison would have been to compare k3s with another distribution like EKS, GKE, OKD etc.
    - jhoelzel 4y ago
      i kindly ask again, what is the distribution called that you setup with kubeadm?
      - glotzerhotze 4y ago
        you might be fooled by the meaning of the word „distribution“ in the kubernetes vs. linux context. in the linux context, a distribution would be an opinionated build of user-land tools around the linux kernel which might also bring patches done to the kernel by the distributor. in the (cloud) kubernetes context, a distribution would be an opinionated (cloud) implementation of core kubernetes binaries (EKS/AKS/etc.pp.) on top of a linux-distribution of your liking. to my humble understanding k3s is a stripped down and optimized (IMHO opinionated) build of kubernetes binaries derived from the official source. kubeadm is the official projects (opinionated?) idea of arriving at a vanilla cluster, leaving you with the freedom to make choices for lots of the needed components involved. in the linux context, „linux from scratch“ would be an analogy I suppose. more knowledgeable people should please correct me if I‘m wrong in my understanding here.
      - moondev 4y ago
        Upstream
paddw 4y ago
K3s seems like a terrible name, given how people will be confused with the numeronym for Kubernetes.
- apetresc 4y ago
  That's the joke, though? It's k8s, but "smaller".
  - mulmen 4y ago
    What are the three letters? It only makes sense for being confusing, which is the base state of Kubernetes. "Our name is literally a complex joke about complexity" doesn't inspire confidence in a project that supposedly simplifies complexity.
    - sakisv 4y ago
      Well, for me the whole replace-letters-with-numbers thing is idiotic and requires some inside knowledge to understand which letters are replaced by all the numbers in words like a11y i19n k8s, etc. On the bright side you can put whatever you want there. Personally I read k3s as "kerts", because why not. Similar to "thirds" but with the "t" from kubernetes.
      - mulmen 4y ago
        I choose to read k3s as "soft boiled egg" because that makes as much sense as anything.
        cwayne 4y ago
        as the manager of the team that maintains it, this is now canon. We will use this exclusively in internal communications. thank you.
    - legerdemain 4y ago
      Kubernetes is often shortened to k8s, which is often pronounced "kates," which is further shortened to k3s.
awinter-py 4y ago
hmmmmm no mention of pvcs eh
- jhoelzel 4y ago
  Longhorn ( https://longhorn.io/ https://longhorn.io/ ) has been stable for a long time now and is also a could native foundation project =) You can start with a single storage node without replication and easily go from there to triple replicated storage
  - ph4te 4y ago
    Been using k3s(kube-vip + MetalLB) and Longhorn for both my personal projects and work projects for a while without any issues. It's a pleasure to use instead of the alternatives.
    - dddw 4y ago
      Interesting combo, care to share more about such a setup?
- docandrew 4y ago
  For simple test/home lab stuff, I’ve had success with this Local Path Provisioner: https://github.com/rancher/local-path-provisioner https://github.com/rancher/local-path-provisioner Just make sure to remember to set it as the default storage class.
gerty 4y ago
Is there any advantage of running k3s if you want to keep etcd? I understand that most k3s performance gains come from etcd being replaced by sqlite but if you still want a HA control plane, sqlite won't cut it.
- iameli 4y ago
  We've been using k3s' embedded etcd for as long as its existed, and it's great. Setting up the etcd cluster is dramatically simplified; let the first node generate a token and feed it to all the other nodes. Tons of other advantages to k3s; the single-binary deploy process, the built-in networking stack (which you can secure with Wireguard out-of-the-box), built-in ingress controller if you want one.
- jhoelzel 4y ago
  you can easily still use external etcd if you really need to. But in general k3s can be HA without issue and scaled just as well as vanilla k8s. The main advantage of it is that everything comes neatly packed into a single binary whereas the alternative would mean to have a multitude of services running for cluster provisioning. Kubernetes in the end is basically an API server with multiple componets and k3s puts a nice bow around all of them.
wouldbecouldbe 4y ago
It was 01:30. About to go to bed. Next day would fly with my daughter to holidays. I checked the apps of a bunch of clients. None of them loaded. I was like what... I checked the server. Everything down. I'd been running Kubernetes on Digital Ocean. And Digital Ocean forced a Kubernetes update that was incompatitable with mine at night. Took me 8 hours to fix it. No sleep. Ended up moving it back to a good old VPS. And throw away K8s. Now to be fair, I had been getting warnings with deployments. But I was used to that, Kubernetes has 10 updates per week. I dont have time to update K8S or my helm files every week. So yeah it was my fault, but I was used to good old VPS hosting. There is an old php application I've build 5 years ago with Laravel, that never needs anything. I did some updates and patches, but it always just works. Im used to running node on apache or nginx, even though a bit less stable, still almost never crashes. Kubernetes there is always something. I guess there are reasons to choose it, but it's not stability. I ended up taking the plane, and my daughter was super kind and patient. But no more kubernetes for me.
- alyandon 4y ago
  My experience so far is that k8s itself is relatively stable. It's when you start using vendor specific addons/plugins to actually do stuff like provision PVs, modify Citrix LB settings, etc outside of k8s everything quickly becomes a burning tire fire. :-/
- jhoelzel 4y ago
  I feel your pain and i would lie if a k8s update never caught me off guard, but there are still policies you can set for the autoupdate of k3s or trigger them manually at will. have a look at k3s and maybe you will like kubernetes more again. There is no magic to it. Have a faulty node? spin up a new one. And if your hoster is kind enough, there will even be apis with cloudinit for you to do that. Im not trying to imply that you need a managed k3s with this post, but rather trying to show how easy kubernetes can be if you leave the big clouds and try not to overcomplicate things.
- bithavoc 4y ago
  DigitalOcean also cordons Nodes for no reason at random times. Currently moving to EKS exactly for this reason
iameli 4y ago
Been using k3s in production for nearly four years now and would recommend it to anyone. Super lightweight and easy to deploy. Opinionated about stuff I don't care about while allowing for customization of network stack, backing database, and ingress controller if you want to do it yourselves. Their embedded etcd is way, way easier to set up than a custom etcd distribution.
- jhoelzel 4y ago
  I have built clusters for clients with k3s too and it has always been a charm. More recently i was able to bring up a Dualstack cluster for a VOIP company that now can basically scale endlessly with gitops on top. The fact that you can integrate existing Sysadmin Teams because they will understand that a program that runs a service with a binary and a config is all it takes, is worth its weight in gold. They know their Loadbalancers and haproxies as well as how to provision true raid systems that are not software based which almost makes disk failure go away and maintenance really sheduleable.
nailer 4y ago
Kinda off topic but what’s the actual word for k3s? We have Kubernetes k8s, addreesen horowitz a16z, internationalisation i18n, founders f6s. What is k3s?
- CGamesPlay 4y ago
  "kubes" -> "k3s"
- deleted 4y ago
  [deleted]
- stonemetal12 4y ago
  It isn't a word. From their documentation: We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10-letter word stylized as K8s. So something half as big as Kubernetes would be a 5-letter word stylized as K3s. There is no long form of K3s and no official pronunciation.
- cwayne 4y ago
  I like to tell people its "kates"
jasoneckert 4y ago
In addition to K3s, I've used managed K8s, custom-rolled K8s, as well as various other K8s distributions. K3s has - by far - provided the least friction for most of my use cases, and is what I incorporate in any initial cloud design. Of course, other stakeholders and constraints may eventually mean that we adopt something else before it gets implemented, but K3s is what I start with for many of the same reasons outlined in this article.
zzyzxd 4y ago
I tried k3s several times in the past few years but I still can't use it in my homelab: 1. the project claims to be production ready and support HA control plane setup, but there's no solution for API load balancing out of box. How do you bring up a new node(either control plane or worker node)? You write down the join token produced by the first control plane node, and hardcode the token and the existing control plane's IP in the new node's systemd unit file. Btw, if you use the official installation script, that file is going to have permission 755 and everyone on the server can just read that token. 2. And how do you bring up the first control plane anyways? The official instruction is to `curl` a bash script and pipe into a shell. You can probably translate that script into some ansible playbook, but the whole running-a-bootstrap-script-and-passing-along-secrets approach make the whole process difficult to be converted into some something that's supposed to be idempotent. All the problems can be worked around, in fact I was half way there, but then I suddenly started thinking: "didn't I choose k3s because I thought it was easy?"
- jhoelzel 4y ago
  Truth be told yes that tricky but can be managed with ansible easily for instance. 1) Your main problem though would probably be the need for a haproxy or bgp which does load balancing for you. There are other solutions like kube-vip but they are more a "failover" solution that HA. Which would be fine for a homelab and is for instance how Rancher Harvester (kubernetes for virtual machines) does it. 2) you have to pass a parameter called --cluster-init for the first node and then join the other nodes. once the cluster is running you dont need any node with that parameter anymore and its common practise to create the first node wiuth --cluster-init then join 3 other ones and take down the first node And on a personal node, you sound like you would be happy with rancher harvester. check it out its bascially turnkey
  - zzyzxd 4y ago
    Yup, when I said that I was half way there, I was trying to use ansible to deploy a nginx systemd unit on every node to load-balancing requests to API servers. Many other distributions can do this with a static pod managed by local kubelet, but that's not possible in k3s. I know there are solutions to my problem, but I can either implement them by myself, or like you mentioned, I can just use a different distribution and not having these problems.
kamikazechaser 4y ago
Title is misleading. k3s is a deployment stack/distribution that builds off various Kubernetes modules. It must pass a certain test suite to conform to Kubernetes standards. What you might be trying to compare is kubeadm which is the official deployment stack provided by Kubernetes.
- jhoelzel 4y ago
  while somewhat yes but no. im not trying to compare it with kubeadm (which is a more a setup script https://kubernetes.io/docs/reference/setup-tools/kubeadm/ https://kubernetes.io/docs/reference/setup-tools/kubeadm/ ) but with the fact that vanilla kubernetes comes with moving parts that have to be configured and maintained and also updated separately. you can actually setup "kubernetes" which is often referred to as vanilla kubernetes without it too. See "Kubernetes the hard way" by kelsey hightower.
- dang 4y ago
  Ok, I've stuffed "default" in the title above. If someone wants to suggest a better (i.e. more accurate and neutral) title, we can change it again.
  - smartbit 4y ago
    I see the term vanilla being used.
    - dang 4y ago
      Ok let's have some vanilla!
alexellisuk 4y ago
I'm a big fan of K3s, however managed Kubernetes from a large cloud vendor with a track record has a lot to offer when it comes to reducing management and the need for an SRE for K8s itself. Folks might also be interested in two free resources: 1 - K3sup https://github.com/alexellis/k3sup https://github.com/alexellis/k3sup - the author mentions HA K3s - K3sup is an easy way to get that using SSH. It's also a good pairing for K3s with Raspberry Pi 2 - Kubernetes at the Edge with K3s (CNCF / LF course) - I was commissioned to write this and I talk a lot about the differences and also the origin story of K3s and what Darren was aiming for. Have fun with Kubernetes - whichever flavour you go for.
wg0 4y ago
I would lean towards k0s over k3s. The article has one piece of information wrong. K8s no more comes with docker. It is containerd I think.
- jhoelzel 4y ago
  the only way i would ever trust a mirantis product again is if they dissolve their board completely. The whole "kubernetes-lens" debacle still burns deep.
  - wg0 4y ago
    Sorry I am totally unaware of that. Any pointer to this lense controversy?
    - jhoelzel 4y ago
      When kubernetes-lens came out it was the greatest software to see what is going on in a clusters. I have suggested it to basically everyone working with k8s saying its a must have tool and also made many teams install it. It was a great time for half a year and a lot of the community contributed to it, only to have them turn around and force accounts on all users: https://github.com/lensapp/lens/issues/5444 https://github.com/lensapp/lens/issues/5444 this turned into an application that now comes with a signed hidden binary, could not be build through the open source repository any more and much ohter things related to this. All in all Mirantis is not to be trusted, not to mention the fact that the tool wants to share your kube-config amongs your team which is a clear antipattern and a high security risk.
moondev 4y ago
I always chuckle to myself when things like k3s, microk8s and so on claim to be "lightweight" kubernetes. Lightweight compared to what exactly? Because pure, upstream, vanilla kubernetes (kubeadm) is the lightest possible, it doesn't come with a CNI, ingress, or any of the additional stuff these distros do. Additionally, why make your life harder by adding an additional layer on top of kubernetes. For troubleshooting then you get to track down if this is a bug in your distro, or actually kubernetes itself. Just run the real thing.
mindwok 4y ago
IMO K3s (and distros like it) are the future of self managed Kubernetes. The same way Linux distributions brought simplification and sane, opinionated defaults to Linux in an era where compiling your own kernel and throwing user space together, K3s does the same for vanilla Kubernetes. It’s a joy to deploy and manage over vanilla.

74 points by jhoelzel 4y ago

the title doesn't make sense, it reads like "comparing Ubuntu with Linux, How Ubuntu is often the better choice". k3s is a kubernetes distribution

ttymck 4y ago
What is the official kubernetes distribution called?
- iameli 4y ago
  There isn't one — the official docs give instructions for kubeadm, kOps, and kubespray. k3s would be an alternative to those.
  - ithkuil 4y ago
    Most of the other K8s distributions use the same K8s binaries. K3s is slightly different in that it's a trimmed down build of the same codebase
  - MuffinFlavored 4y ago
    i think you mean kubeadm and not kubeam
    - iameli 4y ago
      ty, edited
jhoelzel 4y ago
while true, writing "vanialla kubernetes" did not have the right feel for me. What would have been a better title for you?
- user61129028 4y ago
  "vanilla kubernetes" isn't a distribution either. > Kubernetes and k3s are both container orchestration platforms Your article makes it seem like k3s and k8s are different platforms which is simply incorrect. A more useful comparison would have been to compare k3s with another distribution like EKS, GKE, OKD etc.
  - jhoelzel 4y ago
    i kindly ask again, what is the distribution called that you setup with kubeadm?
    - glotzerhotze 4y ago
      you might be fooled by the meaning of the word „distribution“ in the kubernetes vs. linux context. in the linux context, a distribution would be an opinionated build of user-land tools around the linux kernel which might also bring patches done to the kernel by the distributor. in the (cloud) kubernetes context, a distribution would be an opinionated (cloud) implementation of core kubernetes binaries (EKS/AKS/etc.pp.) on top of a linux-distribution of your liking. to my humble understanding k3s is a stripped down and optimized (IMHO opinionated) build of kubernetes binaries derived from the official source. kubeadm is the official projects (opinionated?) idea of arriving at a vanilla cluster, leaving you with the freedom to make choices for lots of the needed components involved. in the linux context, „linux from scratch“ would be an analogy I suppose. more knowledgeable people should please correct me if I‘m wrong in my understanding here.
    - moondev 4y ago
      Upstream

paddw 4y ago

K3s seems like a terrible name, given how people will be confused with the numeronym for Kubernetes.

apetresc 4y ago
That's the joke, though? It's k8s, but "smaller".
- mulmen 4y ago
  What are the three letters? It only makes sense for being confusing, which is the base state of Kubernetes. "Our name is literally a complex joke about complexity" doesn't inspire confidence in a project that supposedly simplifies complexity.
  - sakisv 4y ago
    Well, for me the whole replace-letters-with-numbers thing is idiotic and requires some inside knowledge to understand which letters are replaced by all the numbers in words like a11y i19n k8s, etc. On the bright side you can put whatever you want there. Personally I read k3s as "kerts", because why not. Similar to "thirds" but with the "t" from kubernetes.
    - mulmen 4y ago
      I choose to read k3s as "soft boiled egg" because that makes as much sense as anything.
      - cwayne 4y ago
        as the manager of the team that maintains it, this is now canon. We will use this exclusively in internal communications. thank you.
  - legerdemain 4y ago
    Kubernetes is often shortened to k8s, which is often pronounced "kates," which is further shortened to k3s.

awinter-py 4y ago

hmmmmm no mention of pvcs eh

jhoelzel 4y ago
Longhorn ( https://longhorn.io/ https://longhorn.io/ ) has been stable for a long time now and is also a could native foundation project =) You can start with a single storage node without replication and easily go from there to triple replicated storage
- ph4te 4y ago
  Been using k3s(kube-vip + MetalLB) and Longhorn for both my personal projects and work projects for a while without any issues. It's a pleasure to use instead of the alternatives.
  - dddw 4y ago
    Interesting combo, care to share more about such a setup?
docandrew 4y ago
For simple test/home lab stuff, I’ve had success with this Local Path Provisioner: https://github.com/rancher/local-path-provisioner https://github.com/rancher/local-path-provisioner Just make sure to remember to set it as the default storage class.

gerty 4y ago

Is there any advantage of running k3s if you want to keep etcd? I understand that most k3s performance gains come from etcd being replaced by sqlite but if you still want a HA control plane, sqlite won't cut it.

iameli 4y ago
We've been using k3s' embedded etcd for as long as its existed, and it's great. Setting up the etcd cluster is dramatically simplified; let the first node generate a token and feed it to all the other nodes. Tons of other advantages to k3s; the single-binary deploy process, the built-in networking stack (which you can secure with Wireguard out-of-the-box), built-in ingress controller if you want one.
jhoelzel 4y ago
you can easily still use external etcd if you really need to. But in general k3s can be HA without issue and scaled just as well as vanilla k8s. The main advantage of it is that everything comes neatly packed into a single binary whereas the alternative would mean to have a multitude of services running for cluster provisioning. Kubernetes in the end is basically an API server with multiple componets and k3s puts a nice bow around all of them.

wouldbecouldbe 4y ago

It was 01:30. About to go to bed. Next day would fly with my daughter to holidays. I checked the apps of a bunch of clients. None of them loaded. I was like what... I checked the server. Everything down. I'd been running Kubernetes on Digital Ocean. And Digital Ocean forced a Kubernetes update that was incompatitable with mine at night. Took me 8 hours to fix it. No sleep. Ended up moving it back to a good old VPS. And throw away K8s. Now to be fair, I had been getting warnings with deployments. But I was used to that, Kubernetes has 10 updates per week. I dont have time to update K8S or my helm files every week. So yeah it was my fault, but I was used to good old VPS hosting. There is an old php application I've build 5 years ago with Laravel, that never needs anything. I did some updates and patches, but it always just works. Im used to running node on apache or nginx, even though a bit less stable, still almost never crashes. Kubernetes there is always something. I guess there are reasons to choose it, but it's not stability. I ended up taking the plane, and my daughter was super kind and patient. But no more kubernetes for me.

alyandon 4y ago
My experience so far is that k8s itself is relatively stable. It's when you start using vendor specific addons/plugins to actually do stuff like provision PVs, modify Citrix LB settings, etc outside of k8s everything quickly becomes a burning tire fire. :-/
jhoelzel 4y ago
I feel your pain and i would lie if a k8s update never caught me off guard, but there are still policies you can set for the autoupdate of k3s or trigger them manually at will. have a look at k3s and maybe you will like kubernetes more again. There is no magic to it. Have a faulty node? spin up a new one. And if your hoster is kind enough, there will even be apis with cloudinit for you to do that. Im not trying to imply that you need a managed k3s with this post, but rather trying to show how easy kubernetes can be if you leave the big clouds and try not to overcomplicate things.
bithavoc 4y ago
DigitalOcean also cordons Nodes for no reason at random times. Currently moving to EKS exactly for this reason

iameli 4y ago

Been using k3s in production for nearly four years now and would recommend it to anyone. Super lightweight and easy to deploy. Opinionated about stuff I don't care about while allowing for customization of network stack, backing database, and ingress controller if you want to do it yourselves. Their embedded etcd is way, way easier to set up than a custom etcd distribution.

jhoelzel 4y ago
I have built clusters for clients with k3s too and it has always been a charm. More recently i was able to bring up a Dualstack cluster for a VOIP company that now can basically scale endlessly with gitops on top. The fact that you can integrate existing Sysadmin Teams because they will understand that a program that runs a service with a binary and a config is all it takes, is worth its weight in gold. They know their Loadbalancers and haproxies as well as how to provision true raid systems that are not software based which almost makes disk failure go away and maintenance really sheduleable.

nailer 4y ago

Kinda off topic but what’s the actual word for k3s? We have Kubernetes k8s, addreesen horowitz a16z, internationalisation i18n, founders f6s. What is k3s?

CGamesPlay 4y ago
"kubes" -> "k3s"
deleted 4y ago
[deleted]
stonemetal12 4y ago
It isn't a word. From their documentation: We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10-letter word stylized as K8s. So something half as big as Kubernetes would be a 5-letter word stylized as K3s. There is no long form of K3s and no official pronunciation.
cwayne 4y ago
I like to tell people its "kates"

jasoneckert 4y ago

In addition to K3s, I've used managed K8s, custom-rolled K8s, as well as various other K8s distributions. K3s has - by far - provided the least friction for most of my use cases, and is what I incorporate in any initial cloud design. Of course, other stakeholders and constraints may eventually mean that we adopt something else before it gets implemented, but K3s is what I start with for many of the same reasons outlined in this article.

zzyzxd 4y ago

I tried k3s several times in the past few years but I still can't use it in my homelab: 1. the project claims to be production ready and support HA control plane setup, but there's no solution for API load balancing out of box. How do you bring up a new node(either control plane or worker node)? You write down the join token produced by the first control plane node, and hardcode the token and the existing control plane's IP in the new node's systemd unit file. Btw, if you use the official installation script, that file is going to have permission 755 and everyone on the server can just read that token. 2. And how do you bring up the first control plane anyways? The official instruction is to `curl` a bash script and pipe into a shell. You can probably translate that script into some ansible playbook, but the whole running-a-bootstrap-script-and-passing-along-secrets approach make the whole process difficult to be converted into some something that's supposed to be idempotent. All the problems can be worked around, in fact I was half way there, but then I suddenly started thinking: "didn't I choose k3s because I thought it was easy?"

jhoelzel 4y ago
Truth be told yes that tricky but can be managed with ansible easily for instance. 1) Your main problem though would probably be the need for a haproxy or bgp which does load balancing for you. There are other solutions like kube-vip but they are more a "failover" solution that HA. Which would be fine for a homelab and is for instance how Rancher Harvester (kubernetes for virtual machines) does it. 2) you have to pass a parameter called --cluster-init for the first node and then join the other nodes. once the cluster is running you dont need any node with that parameter anymore and its common practise to create the first node wiuth --cluster-init then join 3 other ones and take down the first node And on a personal node, you sound like you would be happy with rancher harvester. check it out its bascially turnkey
- zzyzxd 4y ago
  Yup, when I said that I was half way there, I was trying to use ansible to deploy a nginx systemd unit on every node to load-balancing requests to API servers. Many other distributions can do this with a static pod managed by local kubelet, but that's not possible in k3s. I know there are solutions to my problem, but I can either implement them by myself, or like you mentioned, I can just use a different distribution and not having these problems.

kamikazechaser 4y ago

Title is misleading. k3s is a deployment stack/distribution that builds off various Kubernetes modules. It must pass a certain test suite to conform to Kubernetes standards. What you might be trying to compare is kubeadm which is the official deployment stack provided by Kubernetes.

jhoelzel 4y ago
while somewhat yes but no. im not trying to compare it with kubeadm (which is a more a setup script https://kubernetes.io/docs/reference/setup-tools/kubeadm/ https://kubernetes.io/docs/reference/setup-tools/kubeadm/ ) but with the fact that vanilla kubernetes comes with moving parts that have to be configured and maintained and also updated separately. you can actually setup "kubernetes" which is often referred to as vanilla kubernetes without it too. See "Kubernetes the hard way" by kelsey hightower.
dang 4y ago
Ok, I've stuffed "default" in the title above. If someone wants to suggest a better (i.e. more accurate and neutral) title, we can change it again.
- smartbit 4y ago
  I see the term vanilla being used.
  - dang 4y ago
    Ok let's have some vanilla!

alexellisuk 4y ago

I'm a big fan of K3s, however managed Kubernetes from a large cloud vendor with a track record has a lot to offer when it comes to reducing management and the need for an SRE for K8s itself. Folks might also be interested in two free resources: 1 - K3sup https://github.com/alexellis/k3sup https://github.com/alexellis/k3sup - the author mentions HA K3s - K3sup is an easy way to get that using SSH. It's also a good pairing for K3s with Raspberry Pi 2 - Kubernetes at the Edge with K3s (CNCF / LF course) - I was commissioned to write this and I talk a lot about the differences and also the origin story of K3s and what Darren was aiming for. Have fun with Kubernetes - whichever flavour you go for.

wg0 4y ago

I would lean towards k0s over k3s. The article has one piece of information wrong. K8s no more comes with docker. It is containerd I think.

jhoelzel 4y ago
the only way i would ever trust a mirantis product again is if they dissolve their board completely. The whole "kubernetes-lens" debacle still burns deep.
- wg0 4y ago
  Sorry I am totally unaware of that. Any pointer to this lense controversy?
  - jhoelzel 4y ago
    When kubernetes-lens came out it was the greatest software to see what is going on in a clusters. I have suggested it to basically everyone working with k8s saying its a must have tool and also made many teams install it. It was a great time for half a year and a lot of the community contributed to it, only to have them turn around and force accounts on all users: https://github.com/lensapp/lens/issues/5444 https://github.com/lensapp/lens/issues/5444 this turned into an application that now comes with a signed hidden binary, could not be build through the open source repository any more and much ohter things related to this. All in all Mirantis is not to be trusted, not to mention the fact that the tool wants to share your kube-config amongs your team which is a clear antipattern and a high security risk.

moondev 4y ago

I always chuckle to myself when things like k3s, microk8s and so on claim to be "lightweight" kubernetes. Lightweight compared to what exactly? Because pure, upstream, vanilla kubernetes (kubeadm) is the lightest possible, it doesn't come with a CNI, ingress, or any of the additional stuff these distros do. Additionally, why make your life harder by adding an additional layer on top of kubernetes. For troubleshooting then you get to track down if this is a bug in your distro, or actually kubernetes itself. Just run the real thing.

mindwok 4y ago

IMO K3s (and distros like it) are the future of self managed Kubernetes. The same way Linux distributions brought simplification and sane, opinionated defaults to Linux in an era where compiling your own kernel and throwing user space together, K3s does the same for vanilla Kubernetes. It’s a joy to deploy and manage over vanilla.