5 ms·

Last time I tried what stumped me was getting "ReadWriteMany" volumes working. My use case was pretty straightforward, a series of processing steps with the ca

Last time I tried what stumped me was getting "ReadWriteMany" volumes working. My use case was pretty straightforward, a series of processing steps with the caveat that storing the actual results in a DB would be bad due to the blobs size. So, instead, the DB is used to signal downstream consumers files are ready and instead write the files in the ReadWriteMany volumes so that downstream files can simply read them. I tried Longhorn, but even if I manage to get a volume showing up in the UI, it was never detected as healthy and after a while I refactored the workflow to use Apache Beam so that I could drop databases and volumes and run everything from within a single beefy VM. Is it still an issue (it was a while ago TBH)?

JeffMcCune 1y ago
Use a bucket for this. Or if you’re in a home lab use a boring old NFS server outside k8s
- marcyb5st 1y ago
  Thanks. The Apache beam code in the end doesn't need a bucket but the HDD attached to the VM was more than enough to store everything and then moving things out of it as the final step.
acedTrex 1y ago
Ya this kind of usecase is for sure straying into the "this is more of a pain than its worth" with regards to k8s. Esp when you can just do basic nfs mounts in nix anyways. If you are already familiar with k8s volumes and csi's its not a huge problem but if you arent its not worth learning if your goal is simple. At least in my opinion.
- marcyb5st 1y ago
  Thank you for taking the time to confirm this. Yeah, that is also my opinion after that experience. Also, it was really frustrating as the number of moving parts that have to play nice to each other is high.
  - acedTrex 1y ago
    K8s and its derivatives ultimately were designed to declaratively orchestrate stateless apis/middlewares. It of course can quite competently do many other things. But it does what it was BUILT to do originally the best and the simplest. Many hobby projects are just that, simple apis and can benefit from the simple declarative yamls that exist in say a sqlite db on the vm vs random config in /etc, /var and ip table scripts etc. But trying to do anything out of that simple mold? just do it the simpler and easier traditional way.

JeffMcCune 1y ago

Use a bucket for this. Or if you’re in a home lab use a boring old NFS server outside k8s

marcyb5st 1y ago
Thanks. The Apache beam code in the end doesn't need a bucket but the HDD attached to the VM was more than enough to store everything and then moving things out of it as the final step.

acedTrex 1y ago

Ya this kind of usecase is for sure straying into the "this is more of a pain than its worth" with regards to k8s. Esp when you can just do basic nfs mounts in nix anyways. If you are already familiar with k8s volumes and csi's its not a huge problem but if you arent its not worth learning if your goal is simple. At least in my opinion.

marcyb5st 1y ago
Thank you for taking the time to confirm this. Yeah, that is also my opinion after that experience. Also, it was really frustrating as the number of moving parts that have to play nice to each other is high.
- acedTrex 1y ago
  K8s and its derivatives ultimately were designed to declaratively orchestrate stateless apis/middlewares. It of course can quite competently do many other things. But it does what it was BUILT to do originally the best and the simplest. Many hobby projects are just that, simple apis and can benefit from the simple declarative yamls that exist in say a sqlite db on the vm vs random config in /etc, /var and ip table scripts etc. But trying to do anything out of that simple mold? just do it the simpler and easier traditional way.