3 ms·
We run petabytes of storage, and growing. We dropped Ceph (which is too complex with a minefield of issues, and few people have deep experience in it) and just
by latchkey 4y ago
We run petabytes of storage, and growing.
We dropped Ceph (which is too complex with a minefield of issues, and few people have deep experience in it) and just use ZFS (which tons of people have experience in and performs flawlessly).
We also use Ansible since it automates and codifies our deployments. I can't imagine dropping it... in favor of what? Manually typing install commands on servers?
It is good they have lifted the covers on their service in this blog post. It screams "stay away".
- arter4 4y agoOut of curiosity, are you running ZFS across datacenters or do you have a single DC? My understanding is, ZFS doesn't inherently provide clustering, so each server has its own pool. If you have multiple DCs, how do you handle disaster recovery? Do you use async replication?
- latchkey 4y agoWe have multiple DC's although the data isn't replicated across DC's. We are a Filecoin storage provider. Users of that network choose where they want to store things and are effectively responsible for geo data replication, then using the protocol we can prove to them that their data is on our machines.
- phphphphp 4y ago> We run petabytes of storage, and growing. That can't be right -- can it? They have less than 50,000 users. For petabytes of storage to be necessary, their average per-user storage would need to be dozens of gigabytes. I've been using GitHub for more than a decade and the sum total of every repository I've published is less than that. I found a few repositories on Codeberg that are using gigabytes of storage but the majority are normal sized Git repositories.
- latchkey 4y agoI'm talking about my own business.