4 ms·

I've run into all of these issues. In the end it's much cheaper to just buy a cheap SAN than it is probably to pay a month of your expertise. I guess that's wh

I've run into all of these issues. In the end it's much cheaper to just buy a cheap SAN than it is probably to pay a month of your expertise. I guess that's why I struggle making cash-cow consulting gigs, because my clients are never long term dependent on me.

gbartolini 3y ago
Why not dedicate some worker nodes using taints/tolerations/labels, even on bare metal, with locally attached storage? I wrote this many years ago now but that's the reason why we started CloudNativePG (OpenEBS might not be the answer today, but there are many storage engines now, including topolvm which brings LVM to the game): https://www.2ndquadrant.com/en/blog/local-persistent-volumes-and-postgresql-usage-in-kubernetes/ https://www.2ndquadrant.com/en/blog/local-persistent-volumes... It is ultimately your choice. I am a big fan of shared nothing architecture for the database. (I am a maintainer of CloudNativePG)
- sebasmannem 3y ago
  Yeah, and let Postgres take care of redundancy. I agree that this is an interesting proposition. AFAIK PortWorkx could do a similar thing, but then with storage redundancy. Basically: - storage is synced to 3 local storage devices spread across 3 different k8s nodes. This could be NVMe. - pod is only scheduled next to one of the three - reads are local, writes are local (for fsync) and synchronised to the other devices. I would love to test with pg_tps_optimizer against Portworkx
sebasmannem 3y ago
I also only run short term. I enable customers with designing and implementing the proper (CEPH e.a.) environment for their workloads. But I don't run their systems. I always handover to the rest of the technical staff. Bringing in a cheap SAN basically shifts that responsibility from people like me to the SAN vendor, and they could bring in hardware that might do the job. Running on K8s, I feel you need two types of storage: - Block storage, with proper fsync (fast and reliable) - S3 storage Both MUST be CloudNative. I don't know if Cheap SAN is available with proper k8s CSI providers, but if they are, they could be up to the challenge. Note that people like me can enable customers with both (choosing the proper 'cheap SAN', but also designing a proper storage environment with CEPH or other software storage solutions.

gbartolini 3y ago

Why not dedicate some worker nodes using taints/tolerations/labels, even on bare metal, with locally attached storage? I wrote this many years ago now but that's the reason why we started CloudNativePG (OpenEBS might not be the answer today, but there are many storage engines now, including topolvm which brings LVM to the game): https://www.2ndquadrant.com/en/blog/local-persistent-volumes-and-postgresql-usage-in-kubernetes/ https://www.2ndquadrant.com/en/blog/local-persistent-volumes... It is ultimately your choice. I am a big fan of shared nothing architecture for the database. (I am a maintainer of CloudNativePG)

sebasmannem 3y ago
Yeah, and let Postgres take care of redundancy. I agree that this is an interesting proposition. AFAIK PortWorkx could do a similar thing, but then with storage redundancy. Basically: - storage is synced to 3 local storage devices spread across 3 different k8s nodes. This could be NVMe. - pod is only scheduled next to one of the three - reads are local, writes are local (for fsync) and synchronised to the other devices. I would love to test with pg_tps_optimizer against Portworkx

sebasmannem 3y ago

I also only run short term. I enable customers with designing and implementing the proper (CEPH e.a.) environment for their workloads. But I don't run their systems. I always handover to the rest of the technical staff. Bringing in a cheap SAN basically shifts that responsibility from people like me to the SAN vendor, and they could bring in hardware that might do the job. Running on K8s, I feel you need two types of storage: - Block storage, with proper fsync (fast and reliable) - S3 storage Both MUST be CloudNative. I don't know if Cheap SAN is available with proper k8s CSI providers, but if they are, they could be up to the challenge. Note that people like me can enable customers with both (choosing the proper 'cheap SAN', but also designing a proper storage environment with CEPH or other software storage solutions.