3 ms·

anyone knows some ressources for running kubernetes outside the cloud?, like guides/best practices/example setups like 1-2 machines scenarios inside airgapped

anyone knows some ressources for running kubernetes outside the cloud?, like guides/best practices/example setups like 1-2 machines scenarios inside airgapped companys, collecting and processing measurements and displaying them, used by managers of industry companies which expect a one click app install etc? i really struggle with kubernetes in that case, and theres barely any documentation/best practices i found it easy to run kubernetes on google,amazon,hetzner, but outside that it feels like hell

threentaway 6y ago
In that case you probably want to be looking at K3s and not full blown Kubernetes.
- gallexme 6y ago
  we are already looking at k3s and microk8s before that we used rancher 1.6 and docker, but rancher 1.6 is dead and was a hell of a software on its own(java extremely high memory usage, obscure bugs, and a team which abandonded it pretty early once 2.0 was released)
tsimionescu 6y ago
My company has similar deployment scenarios as you're describing (basically creating appliances that happen to use K8S as a platform), and we've hit similar problems. It's not (yet?) a very common use case for K8S. Still, so far we haven't found too many problems, and we're using full-blown kubeadm for deployment. Microk8s seemed like a massive dead end when we tried it out, it was very limited and crashy for some reason. Still, we've had a few complicated networking problems that go against some core assumptions of K8S...
cyberpunk 6y ago
We use rancher (actually just rke wrapped in some simple ansible) for this and it works really well. They have a whole guide for running airgapped [0] however, we found that our security requirements (extremely heavily regulated, think nation state and identity) only come into play once the data is ON the systems. So we can happily build a new cluster/environment with internet connections to docker hub or helm.io or whatever, and then as soon as we have data coming in we kill the connectivity -- usually you can wrangle something like this. In either case though, rancher makes running on-prem a really nice experience for us, vs the clusterfsck of kubespray and openshift it replaced. YMMV but I would reccommend giving it a shot. ps -- I would probably avoid K3S if you actually need some redundancy -- unless you're happy managing some kind of distributed mysql setup for it. 0: https://rancher.com/docs/rancher/v2.x/en/installation/other-installation-methods/air-gap/ https://rancher.com/docs/rancher/v2.x/en/installation/other-...
harpratap 6y ago
> collecting and processing measurements and displaying them, used by managers of industry companies which expect a one click app install etc? Kubernetes is just a scheduler/orchestrator. It's not a full blown PaaS as you are expecting it to. You are supposed to glue together parts and make a PaaS based on your needs

threentaway 6y ago

In that case you probably want to be looking at K3s and not full blown Kubernetes.

gallexme 6y ago
we are already looking at k3s and microk8s before that we used rancher 1.6 and docker, but rancher 1.6 is dead and was a hell of a software on its own(java extremely high memory usage, obscure bugs, and a team which abandonded it pretty early once 2.0 was released)

tsimionescu 6y ago

My company has similar deployment scenarios as you're describing (basically creating appliances that happen to use K8S as a platform), and we've hit similar problems. It's not (yet?) a very common use case for K8S. Still, so far we haven't found too many problems, and we're using full-blown kubeadm for deployment. Microk8s seemed like a massive dead end when we tried it out, it was very limited and crashy for some reason. Still, we've had a few complicated networking problems that go against some core assumptions of K8S...

cyberpunk 6y ago

We use rancher (actually just rke wrapped in some simple ansible) for this and it works really well. They have a whole guide for running airgapped [0] however, we found that our security requirements (extremely heavily regulated, think nation state and identity) only come into play once the data is ON the systems. So we can happily build a new cluster/environment with internet connections to docker hub or helm.io or whatever, and then as soon as we have data coming in we kill the connectivity -- usually you can wrangle something like this. In either case though, rancher makes running on-prem a really nice experience for us, vs the clusterfsck of kubespray and openshift it replaced. YMMV but I would reccommend giving it a shot. ps -- I would probably avoid K3S if you actually need some redundancy -- unless you're happy managing some kind of distributed mysql setup for it. 0: https://rancher.com/docs/rancher/v2.x/en/installation/other-installation-methods/air-gap/ https://rancher.com/docs/rancher/v2.x/en/installation/other-...

harpratap 6y ago

> collecting and processing measurements and displaying them, used by managers of industry companies which expect a one click app install etc? Kubernetes is just a scheduler/orchestrator. It's not a full blown PaaS as you are expecting it to. You are supposed to glue together parts and make a PaaS based on your needs