While I totally agree you can easily handle this with Ceph, I would never do it with 2x replication on spinning rust. You can maybe do 2x replication with SSDs

While I totally agree you can easily handle this with Ceph, I would never do it with 2x replication on spinning rust. You can maybe do 2x replication with SSDs if you really know what you're doing and you have data replicated on multiple Ceph clusters. Also, your math does not quite add up if you want to store 425TiB. You'd be filling up the Ceph cluster to 95% which would be impossible (you'll start losing OSDs way before that). If this was me, I would definitely do this on Ceph, but I would opt for a more traditional erasure coding setup of 8+3. For that, you'd need 12x SX134s as a minimum starting point. I suppose you could start with an EC profile of 2+2 or 4+2... I do have a serious question on Hetzner: Do they offer any private networking options? That page only lists a single 1 Gbit connection. I don't think that would work well with Ceph. It's doable, sure, but I wouldn't want to support such a cluster, not with 160TB raw per box.