3 ms·

"Ceph is fine, but its a toy." Because?

KaiserPro 4y ago
Ceph started out as an object storage system with vaguely universal front end slammed on top. It is a good object store, but it's not a file system, and this shows. It has Forward Error Correction scheme that dealt with underlying hardware failure. so your blocks that you store are reasonably safe. But thats the point, its a blockstore/object store. Thats the easy part. allowing concurrent access to the same block, managing write clashes, decent cacheing are hard. Even harder when you are doing it with posix semantics. The main issue with Ceph is that the filesystem is an addon and required a huge amount of resources to make it work properly. (search out the ceph plan for gitlab, before they were bought out by a company that insisted on bringing in sysadmins to sort infra out) the big and largest elephant is that cephfs is still not really "stable" by that I mean supported fully by a large support company. This means that if something goes wrong, you need to figure it out yourself.
- yencabulator 4y ago
  > Ceph started out as an object storage system with vaguely universal front end slammed on top. The opposite. Ceph started out with the underlying load balancing algorithm (May 2004), then with the plan for the filesystem metadata management (Nov 2004). The block storage was created to support the filesystem, virtual block devices were an afterthought to enable private clouds, object storage was an afterthought in a world where S3 got popular. See https://www.ssrc.ucsc.edu/person/sage.html https://www.ssrc.ucsc.edu/person/sage.html for the chronological order. Of course, stabilizing the most complex part of the stack, depending on almost everything else already being stable (everything except RBD and the S3-compatible gateway, which both are relatively slim layers on top of RADOS), takes the longest. Meanwhile, business users were very keen on getting the benefits of RBD for "private clouds" and the object storage, so the filesystem was on the back burner. [Disclaimer: I worked on Ceph pre-RedHat]
lillecarl 4y ago
We have Spectrum at work, it's not comparable. Ceph is great, but Spectrum is a beat and a half!

Ceph started out as an object storage system with vaguely universal front end slammed on top. It is a good object store, but it's not a file system, and this shows. It has Forward Error Correction scheme that dealt with underlying hardware failure. so your blocks that you store are reasonably safe. But thats the point, its a blockstore/object store. Thats the easy part. allowing concurrent access to the same block, managing write clashes, decent cacheing are hard. Even harder when you are doing it with posix semantics. The main issue with Ceph is that the filesystem is an addon and required a huge amount of resources to make it work properly. (search out the ceph plan for gitlab, before they were bought out by a company that insisted on bringing in sysadmins to sort infra out) the big and largest elephant is that cephfs is still not really "stable" by that I mean supported fully by a large support company. This means that if something goes wrong, you need to figure it out yourself.

yencabulator 4y ago
> Ceph started out as an object storage system with vaguely universal front end slammed on top. The opposite. Ceph started out with the underlying load balancing algorithm (May 2004), then with the plan for the filesystem metadata management (Nov 2004). The block storage was created to support the filesystem, virtual block devices were an afterthought to enable private clouds, object storage was an afterthought in a world where S3 got popular. See https://www.ssrc.ucsc.edu/person/sage.html https://www.ssrc.ucsc.edu/person/sage.html for the chronological order. Of course, stabilizing the most complex part of the stack, depending on almost everything else already being stable (everything except RBD and the S3-compatible gateway, which both are relatively slim layers on top of RADOS), takes the longest. Meanwhile, business users were very keen on getting the benefits of RBD for "private clouds" and the object storage, so the filesystem was on the back burner. [Disclaimer: I worked on Ceph pre-RedHat]

lillecarl 4y ago

We have Spectrum at work, it's not comparable. Ceph is great, but Spectrum is a beat and a half!