4 ms·
Does anyone have good experiences with a replicated storage CSI which can run on commodity hardware? I tried out OpenEBS Replicated — and it is promising — but
by __turbobrew__ 2y ago
Does anyone have good experiences with a replicated storage CSI which can run on commodity hardware?
I tried out OpenEBS Replicated — and it is promising — but it doesn’t really seem mature yet. Im a bit scared to put production critical data into it.
- candiddevmike 2y agoWhat problem are you trying to solve with replicated storage? A lot of times, finding a solution further up the stack or settling for backups ends up being more robust and reliable. Many folks have been burnt by all the fun failure scenarios of replicated filesystems.
- __turbobrew__ 2y agoI build a platform which hosts a few thousand services and tens of thousands of nodes. Currently, those services need to each internally manage replicating data and need to be aware of failure domains (host, rack, data center). What I would like to do is develop a system where applications just need to request replicated volumes which span a specific failure domain and push that logic down to the platform.
- iampims 2y agoRook would suit your problem space very well.
- __turbobrew__ 2y agoThank you
- noctarius 2y agoGuess you want a block storage?
- erulabs 2y agoMy experience is that OpenEBS and Longhorn are cool and new and simplified, but that I would only trust my life to Rook/Ceph. If it's going into production, I'd say look at https://rook.io/ https://rook.io/ - Ceph can do both block and filesystem volumes.
- __turbobrew__ 2y agoThanks, I will look at Rook.
- withinboredom 2y agoLonghorn, hands down. It’s dead simple to set up and works well with production workloads. We’ve had disks fail, nodes fail, etc. and it has handled everything brilliantly. It’s also near-native speeds, which is really nice.
- ngharo 2y agoNear native speeds? I’m seeing an order of magnitude slower performance out of mine. Doing anything special with your config? I already am setting placement options and played with replica options. My only hope has been to wait for the V2 engine to become stable.
- withinboredom 2y agoI guess it depends on how you are measuring it! If you compare it to running it on a RAID5, it is. We are running it on a RAID0 and using Longhorn replication to provide replication instead of the RAID and using striping to get more throughput.