3 ms·
I've used LVM in AWS to present a single volume to Kafka >16TB which is their max EBS size. They report you can attach up to 40 EBS volumes. Growing LVM with X
by beepbeepbeep1 8y ago
I've used LVM in AWS to present a single volume to Kafka >16TB which is their max EBS size. They report you can attach up to 40 EBS volumes.
Growing LVM with XFSs has worked well, 0 downtime and around 60 seconds.
Allows you to over provision just enough you do not have to babysit the drives or pay $$$ for unused disc.
If you stripe the volumes you'll also distribute your IOPs in AWS.
Outside AWS LVM still applies. Kafka's JBOD is useless without easy / auto rebalancing.
This week onsite at a client's I discovered ScaleIO which can present up to a 1PB volume and does clever sharding/replication in the background.