4 ms·

Hey, thanks for all the good ideas. About the custom "VM-only" machine: currently I'm a little lacking in my "k8s ops" skills. So, until I am more comfortable

by 0xC45 6y ago

Hey, thanks for all the good ideas. About the custom "VM-only" machine: currently I'm a little lacking in my "k8s ops" skills. So, until I am more comfortable managing the k8s environment, I have decided to continue using VMs for my "production" workloads. Until then, the K8s cluster will be treated like more of an experimental zone and I can worry less about breaking things. As for the K8s storage options -- I don't really know! Haven't quite figured that one out. Like I mentioned in the blog post, I haven't really placed any serious workload on the K8s cluster yet. However, I have heard good things about OpenEBS (https://openebs.io/ https://openebs.io/) and am considering that as a potential option in order to make use of the local SSDs on each NUC. For single-node K8s, there are lots of good options. I haven't used it personally, but I know that MicroK8s (https://microk8s.io/ https://microk8s.io/) works well even in a single-node environment. It might be worth checking out.

DenseComet 6y ago
I'm running around 90% of the things in my homelab in a kubernetes cluster. I love it, but storage has been the biggest challenge. From my experience, OpenEBS and Rook both seem to be good options when you have multiple physical machines with disks on each one. Portworx is also very good, but the licensing costs are way too high. I'm running my kubernetes nodes as VMs and I didn't want to have to backup the virtual machine disks. I just wanted to back up my freenas server, so I went with trying to use NFS. I wanted to use the NFS client provisioner, as I thought I'd want to be able to add and remove volumes automatically, however, this makes redeploying onto a new cluster from scratch hard. Eventually, I just tore that out and just mounted each NFS volume individually because it turns out that I actually only need a handful of volumes.
- dijit 6y ago
  Maybe useful information, I'm not sure: but I was at DellEMC world in 2017 and they showed off this really cool k8s storage tech. It was called RexRay and was obviously being demo'd with ScaleIO in mind. However I see that it has multiple backends (including CEPH), might be worth looking into as it seemed very fault tolerant and fast. :) https://github.com/rexray/rexray https://github.com/rexray/rexray
tyldum 6y ago
For simple storage, rancher longhorn is also an option.
- johntash 6y ago
  +1 for Longhorn. I started using it in my homelab as an easy replicated/distributed storage. It was a lot easier to setup and had more of an "it just works" out of the box experience compared to openebs/rook/etc.

by 0xC45 6y ago

DenseComet 6y ago

I'm running around 90% of the things in my homelab in a kubernetes cluster. I love it, but storage has been the biggest challenge. From my experience, OpenEBS and Rook both seem to be good options when you have multiple physical machines with disks on each one. Portworx is also very good, but the licensing costs are way too high. I'm running my kubernetes nodes as VMs and I didn't want to have to backup the virtual machine disks. I just wanted to back up my freenas server, so I went with trying to use NFS. I wanted to use the NFS client provisioner, as I thought I'd want to be able to add and remove volumes automatically, however, this makes redeploying onto a new cluster from scratch hard. Eventually, I just tore that out and just mounted each NFS volume individually because it turns out that I actually only need a handful of volumes.

dijit 6y ago
Maybe useful information, I'm not sure: but I was at DellEMC world in 2017 and they showed off this really cool k8s storage tech. It was called RexRay and was obviously being demo'd with ScaleIO in mind. However I see that it has multiple backends (including CEPH), might be worth looking into as it seemed very fault tolerant and fast. :) https://github.com/rexray/rexray https://github.com/rexray/rexray

tyldum 6y ago

For simple storage, rancher longhorn is also an option.

johntash 6y ago
+1 for Longhorn. I started using it in my homelab as an easy replicated/distributed storage. It was a lot easier to setup and had more of an "it just works" out of the box experience compared to openebs/rook/etc.