34 ms·

Hello, just out of curiosity, is using the NFS Client Provisioner Helm chart necessary anymore for normal kubernetes deployments? My company was using it, but i

Hello, just out of curiosity, is using the NFS Client Provisioner Helm chart necessary anymore for normal kubernetes deployments? My company was using it, but it became a bit of a hassle on GKE for our team, as we found that A) if you tried to reprovision the NFS Client Provisioner then it would no longer mount the NFS share directly, it would create a new subdirectory and the files in the top level of the NFS share would be inaccessible within our pods, and B) It did not support NFSv4. After some research we found that Kubernetes is able to utilize NFS shares natively these days. You can either mount a NFS share directly in a pod as long as its reachable in the network: https://docs.docker.com/ee/ucp/kubernetes/storage/use-nfs-volumes/#defining-nfs-shares-in-the-pod-definition https://docs.docker.com/ee/ucp/kubernetes/storage/use-nfs-vo... However, if you still would like to pre-provision the NFS share as a PV/PVC you are also able to do that and expose the NFS mount as a Kubernetes Object. https://docs.docker.com/ee/ucp/kubernetes/storage/use-nfs-volumes/#exposing-nfs-shares-as-a-cluster-object https://docs.docker.com/ee/ucp/kubernetes/storage/use-nfs-vo... Just wondering, now that NFS mounts are directly available within Kubernetes, is there any advantages to utilizing a "3rd party" NFS Client Provisioner Helm deployment? Anyways, looks like a really great write up. Although Kubernetes can be a big hassle for small teams, I feel like its slowly becoming more and more user accessible and user friendly to more organizations. Even with the mental overhead and hassle of running kubernetes, my team at work has seen some clear benefits from going to an ec2/ansible based architecture to a GKE/helm based architecture.

xiwenc 6y ago
You are right the NFS client provisioner is not needed. This approach was chosen because the NFS server address only has to be configured once for the provisioner. Each PVC claims a subfolder within the shared NFS share. This might be undesirable for certain scenarios, but it works for us with zero friction. Creating a PVC is as simple as: --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: data spec: accessModes: - ReadWriteMany resources: requests: storage: 5Gi And because of this, it's compatible with pretty much all helm charts that rely on the default StorageClass that works without extra configurations or manual operations like creating shares and figuring out what NFS server to use.
DenseComet 6y ago
I've been using the NFS client provisioner for my lab k8s cluster for a while. The reason I decided to use it was because I wanted to avoid having to create the shares for that service manually. I thought this would be useful for StatefulSets, which provisions a single volume for every pod, letting the application manage replication. After a while, I've realized that it was a bad idea, because it turns out I don't run many services so creating a share for each service is a big deal, and that storing the StatefulSet volumes on a single server basically defeats the purpose of StatefulSets. The second problem was that when I deployed it to a new cluster, I would get new paths on the NFS server, so the data from a previous cluster was not recognized. Since it is a lab environment, it is better for me to not bother with backing up the k8s cluster. With set paths, I can backup the NFS server, and redeploy a clean k8s cluster from the manifests and have the pods recognize previously written data. As far as I can tell, the provisioner is mostly beneficial if you need to create volumes on the fly (although its possible that this is also possible with directly mounting NFS shares). On another note, have you looked at using one of the many ways of using persistent storage with GKE that isn't NFS? It seems that Google has created connectors and stuff for their own cloud storage that might be more straightforward to use.

xiwenc 6y ago

You are right the NFS client provisioner is not needed. This approach was chosen because the NFS server address only has to be configured once for the provisioner. Each PVC claims a subfolder within the shared NFS share. This might be undesirable for certain scenarios, but it works for us with zero friction. Creating a PVC is as simple as: --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: data spec: accessModes: - ReadWriteMany resources: requests: storage: 5Gi And because of this, it's compatible with pretty much all helm charts that rely on the default StorageClass that works without extra configurations or manual operations like creating shares and figuring out what NFS server to use.

DenseComet 6y ago

I've been using the NFS client provisioner for my lab k8s cluster for a while. The reason I decided to use it was because I wanted to avoid having to create the shares for that service manually. I thought this would be useful for StatefulSets, which provisions a single volume for every pod, letting the application manage replication. After a while, I've realized that it was a bad idea, because it turns out I don't run many services so creating a share for each service is a big deal, and that storing the StatefulSet volumes on a single server basically defeats the purpose of StatefulSets. The second problem was that when I deployed it to a new cluster, I would get new paths on the NFS server, so the data from a previous cluster was not recognized. Since it is a lab environment, it is better for me to not bother with backing up the k8s cluster. With set paths, I can backup the NFS server, and redeploy a clean k8s cluster from the manifests and have the pods recognize previously written data. As far as I can tell, the provisioner is mostly beneficial if you need to create volumes on the fly (although its possible that this is also possible with directly mounting NFS shares). On another note, have you looked at using one of the many ways of using persistent storage with GKE that isn't NFS? It seems that Google has created connectors and stuff for their own cloud storage that might be more straightforward to use.