8 ms·

Anyone know how this compares to k3s? I’ve been using k3s for a while, and there have been a few bugs that made me a bit annoyed

• ivan_ah 6y ago
  Same question, but with minikube.
  • koeng 6y ago
    I think minikube is for local usage. I use K3S for deploying an actual production cluster, so I’m more curious about how micro-k8s acts on that scale
    • thor24 6y ago
      > I use K3S for deploying an actual production cluster You mean when you want to run a small cluster of let's say less than 10 nodes (anything in single digits)? Why doesn't normal k8 work this way? like same tech but just less scale? (I should probably read more on the side myself as well, new to this K8 world).
      • koeng 6y ago
        Yep, it’s a small self hosted cluster on my own server hardware (VMs on Proxmox) I’m sure k8s would work, but k3s made it really easy for me to get going so now just has inertia.
        • thor24 6y ago
          I see, I assumed logically Kubernetes as a community would want to keep thing simple and keep it same across scale. For large scale there could be different variations (like switching storage backend) but for small/low scale tier keep it dead simple. I am guessing there were valid reasons for the offshoots (k3s, microk8s etc.).
      • moondev 6y ago
        It does. Upstream kubeadm - authentic kubernetes - can even run on a single node. Not sure why people choose k3s or microk8s when you can just as easily deploy the real thing.
        • geerlingguy 6y ago
          For smaller needs, K3s runs fast/stable on servers with 1-2 GB of RAM, whereas K8s proper tends to be a little shaky until you go to 2-4 GB, minimum.
          • moondev 6y ago
            I hear this a lot but have any actual, tangible comparisons been done with k3s vs kubeadm arm resource use?
        • wichert 6y ago
          kubeadm is very bare bones though; it gives you a running node but you are still responsible for configuring network and storage providers, an ingress controller and probably a load balancer.
        • thor24 6y ago
          That's exactly what i am confused about when only reading these things in isolation.
• rektide 6y ago
  The first thing I think of with microk8s is their available addons, which are fairly comprehensive, opinionated, "just works" solutions for microk8s. https://microk8s.io/docs/addons#heading--list https://microk8s.io/docs/addons#heading--list As a k3s user (for local dev & my very small personal prod envs), I end up having to assemble a lot of these solutions myself. I prefer my own picks, my own solutions, and learning, but microk8s having these instantly available would be really good for a lot of folks. Until now though it has never felt like those advantages would be useful in a real prod environment, that microk8s was not interested in being in prod, but HA signals to me that they are interested in broader adoption.
• pmahoney 6y ago
  This doesn't answer your question, but I'm piggy-backing with comments on k3s. For me, using k3s for development (not prod), the killer feature is running it in --docker mode where the node uses the local dockerd to run containers (vs. managing its own containerd instance). This allows building images locally with `docker build` and immediately using them in kubernetes pods _without_ first pushing to a (possibly local) image repository and having k3s pull the images. Last time I investigated, none of kind, microk8s, or minikube supported this mode. For large images (gigabyte or more, and I've got a handful of these), it's very space-inefficient to have a copy in my docker and in a local registry _and_ in k3s's containerd at the same time. How is this problem typically solved? (I note the kubernetes included in Docker Desktop on macOS works in the same way: images built with `docker build` are available to kubernetes without going through a registry.)
  • moduspol 6y ago
    We're using img [1] with a helper script. The helper script runs a pod in the (potentially) remote Kubernetes cluster that takes the container build context as stdin, then runs img to build the image, and then imports it into the host's containerd instance. It also maps in the right host volumes so that the image build is cached between subsequent runs. I was surprised to be unable to otherwise find a good local / remote container development workflow, but this was built to replace what we were previously doing, which is setting DOCKER_HOST to point to the remote (single-node) cluster's Docker daemon (over SSH), so that docker CLI commands would execute on that remote box. In both cases, you'll still want to take steps to minimize the size of your container image build context, but the size of the images doesn't matter. I'm not sure if it'd fit your needs or not, though. [1] https://github.com/genuinetools/img https://github.com/genuinetools/img
  • TeamTeaTime 6y ago
    FYI minikube supports this via minikube docker-env https://minikube.sigs.k8s.io/docs/handbook/pushing/#1-pushing-directly-to-the-in-cluster-docker-daemon-docker-env https://minikube.sigs.k8s.io/docs/handbook/pushing/#1-pushin...
  • bryanlarsen 6y ago
• candiddevmike 6y ago
  It uses distributed sqlite (dqlite) vs etcd/RDBMS (postgresql and friends). It's also not part of the CNCF like k3s is (yet?).
  • timClicks 6y ago
    dqlite is a RDBMS, it's just sqlite + raft
• hinkley 6y ago
  k3s swaps etcd for kine, which is an app that implements enough of the etcd API to support kubernetes. It uses dqlite under the hood, but you can use a different database if you want (though it's not clear to me if you still get Raft if you do so). It looks like Canonical is just using dqlite directly.
• sandGorgon 6y ago
  I'm running k3s in production. K3s has hooks to setup Prometheus, autoscaling (for spot instances), etc. I don't see all of these in microk8s. I'm not sure if this is on the roadmap. I'm also not sure how customisable microk8s is. We run k3s with haproxy ingress (which is not the default) and calico for network (again nog the default)
  • merb 6y ago
    sadly you need to run with kube-proxy and can't use calico's own sauce. but I already raised an issue for that, if I would use it more instead of kubespray (which I will deprecate soon) than I maybe fix it on my own time.
  • ericflo 6y ago
    microk8s has a Prometheus add-on that you can enable with one command: https://microk8s.io/docs/addons https://microk8s.io/docs/addons I'm using it and it's been great so far.
• elderbarry 6y ago
  Several commenters here say they have fewer bugs with MicroK8s, makes sense since it’s less of a modification than k3s. Slightly bigger since it keeps api and worker processes as separate binaries albeit in a single package.