Not the OP, but I've been running production Ceph clusters for the past 4.5 years at two different fortune-50 companies. We've had very good success with Ceph

Not the OP, but I've been running production Ceph clusters for the past 4.5 years at two different fortune-50 companies. We've had very good success with Ceph for block storage and a fairly rough time with it for object storage. We're currently doing our best to try to improve it (both our own upstream contributions and collaborating with RedHat). From a technology standpoint, I think it is very interesting and for the most part has a lot of very good engineering. However, it is fairly complex and even today it's very easy to have a hard time with it when starting out. You really need to pay attention to every detail and your hardware selection is extremely important. It is extremely resilient, and goes to great lengths to preserve your data. Ceph can be performant, however that requires very good hardware and network. My experience is limited up to the Jewel release (we haven't upgraded to Luminous and we are not planning on using BlueStore anytime soon).