3 ms·

When I’m reading about Kubernetes, it makes me tremble. Like you need three servers with 16 GB RAM each just for some management stuff. I have one server with

When I’m reading about Kubernetes, it makes me tremble. Like you need three servers with 16 GB RAM each just for some management stuff. I have one server with 1 GB RAM. Can I start with that? And expand on more servers later if need arises.

gndoped 5y ago
I think k3d would be a good choice for you as others have mentioned. It's a lightweight wrapper on top of k3s - https://k3d.io/v5.2.2/ https://k3d.io/v5.2.2/
guywhocodes 5y ago
Yes with 1GB you can run k3s.
Bayart 5y ago
Yes you can, that's the point really. You can just start off on your own machine with something like Minikube [1] and deploy to GCP on a bunch of much larger nodes to see how it behaves. The overhead of Kubernetes itself isn't much, it's just a bunch of API daemons. For small scale development, plain docker/podman is better in my opinion. [1]: https://minikube.sigs.k8s.io/docs/start/ https://minikube.sigs.k8s.io/docs/start/
KronisLV 5y ago
Seconding K3s being a good option, supports most distros and is easy to set up in minutes: https://k3s.io/ https://k3s.io/ If you'd like to use a management UI instead of just the CLI, Portainer is also a wonderfully lightweight option: https://www.portainer.io/ https://www.portainer.io/ (and also supports Docker Swarm, if you'd like even more lightweight container orchestration) That said, in my eyes the biggest problem with Kubernetes is that some distros by default reserve resources and thus disallow overcommit, which is fine from a stability perspective, but doesn't really work for overcommit scenarios. For example, with Docker Swarm, it's exceedingly easy to say: "Okay, this PostgreSQL instance can use anywhere from 0 to 256 MB of RAM, but no more than that" and with 1 GB of RAM i could easily run 8 of those instances if the average usage was half or less of this maximum limit - a risk that i'd sometimes like to take, when i know that the limits in place are more along the lines of controls so that the whole OS doesn't run into OOM errors. Of course, if you mess around with the YAML, i guess that's doable on some level as well. As a sidenote: most of the software that i run (Java, .NET, Ruby, Python, Node, Go, PHP) all is memory constrained most of the times, so when looking for server specs, i usually prioritized the memory the highest, CPU, storage and network speed all usually being a secondary concern. Sadly, in the current day and age, i'm not sure how far you could actually get with 1 GB or why even use container orchestration at that point. The smallest VPSes that i have come with 2 GB of RAM and even those are for smaller pages/proxies and such, whereas the majority have at least 4 GB (since GitLab and Nexus won't launch and work with much less) and my homelab servers have 16 GB, whereas my workstation has 24 GB at this point, because that's all that i can afford. Wirth's law be damned, i doubt that even OSes will work well with that much in the future, especially because of desktop software being infected with Electron and server software having JVM eat as much memory as you'll give it (though that also happens with MySQL or MariaDB). I don't care about "Unused RAM is wasted RAM" as much as others do, i just want to run more stuff on my servers and to have software that doesn't NEED the memory not take it.
kastden 5y ago
I'd suggest trying out https://k3s.io/ https://k3s.io/ then. Runs on pretty much anything and supports multiple nodes that can be added later. They have ripped out a lot of things you might not need (cloud drivers, etc). Despite that, it was easy to install things like storage drivers or Cilium for networking when I wanted that. I used k3sup to setup my cluster, but that was just for convenience.
El_RIDO 5y ago
Yes, you can start with that. There is a k8s distro called k3s that runs fine even on 1GB Raspberry Pi 3. I have been running it on those for a while, but also on other SBCs. You can grow the single master by attaching one or more workers to it. The "pain" is mainly that you can't expand the single-master to a multi-master setup, so once you want to go beyond a single master, you have to re-install it all. On the other hand, with k8s you have all deployments, etc. in yaml files and you simply redeploy these on the new cluster. My remaining pain-point is persistent storage - it's simple to run complex setups with stateless containers or that can retrieve there data at initialization (i.e. a secondary name server that populates itself from the master at startup), but for classic databases I currently still have a manual step to import the DB dump once the service got (re)started/migrated. Sources: - https://k3s.io/ https://k3s.io/ - https://www.jeffgeerling.com/blog/2020/installing-k3s-kubernetes-on-turing-pi-raspberry-pi-cluster-episode-3 https://www.jeffgeerling.com/blog/2020/installing-k3s-kubern...
cogman10 5y ago
Where on earth are you getting that notion? K8s operates just fine on 1GB nodes. It's very low overhead. You can operate it on a single node cluster, but that's generally not recommended, not for performance reasons, but for redundancy. The reason larger nodes are recommended is that k8s is for fleets of applications. If you are only talking about a single service, then a k8s cluster is overkill. K8s is the cheapest way to run a fleet of microservices.
mrweasel 5y ago
I believe you can get away with much less, but this is something that concerns me as well. We deal with customers that want Kubernetes, but the management plane uses more servers/VMs than is required to run their code. There is mini-Kubernetes implementation, but I don't know if they're any good for production. There seems to be a limit to how small you can go with Kubernetes in my mind, and most people never need to cross the line where they're large enough that Kubernetes makes sense. It's honestly a niche product, but right now it's being viewed as a catch all solution.

gndoped 5y ago

I think k3d would be a good choice for you as others have mentioned. It's a lightweight wrapper on top of k3s - https://k3d.io/v5.2.2/ https://k3d.io/v5.2.2/

guywhocodes 5y ago

Yes with 1GB you can run k3s.

Bayart 5y ago

Yes you can, that's the point really. You can just start off on your own machine with something like Minikube [1] and deploy to GCP on a bunch of much larger nodes to see how it behaves. The overhead of Kubernetes itself isn't much, it's just a bunch of API daemons. For small scale development, plain docker/podman is better in my opinion. [1]: https://minikube.sigs.k8s.io/docs/start/ https://minikube.sigs.k8s.io/docs/start/

KronisLV 5y ago

Seconding K3s being a good option, supports most distros and is easy to set up in minutes: https://k3s.io/ https://k3s.io/ If you'd like to use a management UI instead of just the CLI, Portainer is also a wonderfully lightweight option: https://www.portainer.io/ https://www.portainer.io/ (and also supports Docker Swarm, if you'd like even more lightweight container orchestration) That said, in my eyes the biggest problem with Kubernetes is that some distros by default reserve resources and thus disallow overcommit, which is fine from a stability perspective, but doesn't really work for overcommit scenarios. For example, with Docker Swarm, it's exceedingly easy to say: "Okay, this PostgreSQL instance can use anywhere from 0 to 256 MB of RAM, but no more than that" and with 1 GB of RAM i could easily run 8 of those instances if the average usage was half or less of this maximum limit - a risk that i'd sometimes like to take, when i know that the limits in place are more along the lines of controls so that the whole OS doesn't run into OOM errors. Of course, if you mess around with the YAML, i guess that's doable on some level as well. As a sidenote: most of the software that i run (Java, .NET, Ruby, Python, Node, Go, PHP) all is memory constrained most of the times, so when looking for server specs, i usually prioritized the memory the highest, CPU, storage and network speed all usually being a secondary concern. Sadly, in the current day and age, i'm not sure how far you could actually get with 1 GB or why even use container orchestration at that point. The smallest VPSes that i have come with 2 GB of RAM and even those are for smaller pages/proxies and such, whereas the majority have at least 4 GB (since GitLab and Nexus won't launch and work with much less) and my homelab servers have 16 GB, whereas my workstation has 24 GB at this point, because that's all that i can afford. Wirth's law be damned, i doubt that even OSes will work well with that much in the future, especially because of desktop software being infected with Electron and server software having JVM eat as much memory as you'll give it (though that also happens with MySQL or MariaDB). I don't care about "Unused RAM is wasted RAM" as much as others do, i just want to run more stuff on my servers and to have software that doesn't NEED the memory not take it.

kastden 5y ago

I'd suggest trying out https://k3s.io/ https://k3s.io/ then. Runs on pretty much anything and supports multiple nodes that can be added later. They have ripped out a lot of things you might not need (cloud drivers, etc). Despite that, it was easy to install things like storage drivers or Cilium for networking when I wanted that. I used k3sup to setup my cluster, but that was just for convenience.

El_RIDO 5y ago

Yes, you can start with that. There is a k8s distro called k3s that runs fine even on 1GB Raspberry Pi 3. I have been running it on those for a while, but also on other SBCs. You can grow the single master by attaching one or more workers to it. The "pain" is mainly that you can't expand the single-master to a multi-master setup, so once you want to go beyond a single master, you have to re-install it all. On the other hand, with k8s you have all deployments, etc. in yaml files and you simply redeploy these on the new cluster. My remaining pain-point is persistent storage - it's simple to run complex setups with stateless containers or that can retrieve there data at initialization (i.e. a secondary name server that populates itself from the master at startup), but for classic databases I currently still have a manual step to import the DB dump once the service got (re)started/migrated. Sources: - https://k3s.io/ https://k3s.io/ - https://www.jeffgeerling.com/blog/2020/installing-k3s-kubernetes-on-turing-pi-raspberry-pi-cluster-episode-3 https://www.jeffgeerling.com/blog/2020/installing-k3s-kubern...

cogman10 5y ago

Where on earth are you getting that notion? K8s operates just fine on 1GB nodes. It's very low overhead. You can operate it on a single node cluster, but that's generally not recommended, not for performance reasons, but for redundancy. The reason larger nodes are recommended is that k8s is for fleets of applications. If you are only talking about a single service, then a k8s cluster is overkill. K8s is the cheapest way to run a fleet of microservices.

mrweasel 5y ago

I believe you can get away with much less, but this is something that concerns me as well. We deal with customers that want Kubernetes, but the management plane uses more servers/VMs than is required to run their code. There is mini-Kubernetes implementation, but I don't know if they're any good for production. There seems to be a limit to how small you can go with Kubernetes in my mind, and most people never need to cross the line where they're large enough that Kubernetes makes sense. It's honestly a niche product, but right now it's being viewed as a catch all solution.