Unless you have serious simulation or model checking tools, your ad hoc combination of etcd and ceph will almost certainly be buggy. I'm not 100% positive it is

Unless you have serious simulation or model checking tools, your ad hoc combination of etcd and ceph will almost certainly be buggy. I'm not 100% positive it is even possible in the asynchronous model. And when you decide you want, say, regional or rack aware fault tolerance, you have three more distributed systems (ceph, etcd, and your combination) to make meet that goal. When you restore a backup of the contents of ceph, what do you do with the lock state in etcd? Do you know what happens when (say) etcd runs out of memory? If you wanted to ship an on premise version of your solution, can you teach your customers to admin all these systems? FDB's mission is to get under all of your state storage so that you only need one system to do all this operational stuff correctly. A block device is most useful for people who are trying to eliminate the last bits of state not in FDB from their infrastructure. Another huge payoff is the new "satellite" replication mode coming (I'm told) with FDB 6.0, which will give you "the best of both worlds" of synchronous and asynchronous replication across regions. A block device will let your legacy systems usually fail over across regions automatically without losing a single write.