4 ms·
We're not doing anything explicitly posix with ceph - not using the ceph filesystem. We're using ceph for block storage under virtual machines that use KVM/QEMU
by andyidsinga 12y ago
We're not doing anything explicitly posix with ceph - not using the ceph filesystem. We're using ceph for block storage under virtual machines that use KVM/QEMU with libvirt etc. The block devices exposed to the VMs end up using having an ext4 filesystem on them but that has nothing to do with ceph.
So far seems fantastic from a VM perspective -- I can take down storage nodes, add new nodes etc, all while the VMs running up top stay alive and well. I once lost 1/3 of my cluster (on a 3 node cluster with 30 disks) ..and everything stayed running until I fixed the broken node.
Ceph also supports Object storage and an S3 interface -- I haven't experimented much with that yet, but I hear its good.
- xorcist 12y agoThanks, that sounds like a very interesting use case. It sounds almost perfect for KVM storage. How do you expose the file system to the KVM nodes? Just mount it? What's performance like so far?
- andyidsinga 12y agoqemu has support for ceph's "rbd" ( rados block device). you create an image of a particular size using the ceph tools, then when you create a vm xml definition you specify and rbd disk. ( see here http://libvirt.org/formatdomain.html#elementsDisks http://libvirt.org/formatdomain.html#elementsDisks ) finally, when the vm is booted you can use that block device like any other machine. for first install it will be empty ( need to attach a cdrom image to get going ) then youncan start making snapshots for "thin provisioning" using copy on write clones ...very sweet stuff :) re performance ...i think its good, but i have heard its not as good as expensive proprietary systems. the product we're building is around storage analytics on open source. ..so eventually well know more :)