11 ms·
I'd love to know how Apple distributes its CloudKit data across all of (what I assume are) its dozens of globally distributed FoundationDB clusters. As of vers
by OctoberTime 5y ago
I'd love to know how Apple distributes its CloudKit data across all of (what I assume are) its dozens of globally distributed FoundationDB clusters.
As of version 6.0, a single FoundationDB cluster currently only supports one active, writable 'master' region + one hot failover region. Transactions are all essentially LAN so performance characteristics are great. It doesn't support the slower TiDB/CRDB style active/active configuration where all nodes are writable, data is globally distributed and transactions can span across all nodes in the cluster.
Do they have some kind of scheme whereby ACID is preserved for local transactions only, while far away regions are asynchronously replicated to with less guarantees on data consistency?
- deleted 5y ago[deleted]
- adamfeldman 5y agoThere were 2 FDB papers published recently – this one discusses some of that: https://www.foundationdb.org/files/QuiCK.pdf https://www.foundationdb.org/files/QuiCK.pdf
- Operyl 5y agoI suspect it’s because they’re not just running a single cluster, but many many clusters with customers distributed globally. They had a talk about how they do it with other systems some time ago that I can’t find at the moment.
- foobiekr 5y agoMaximum fdb cluster size is actually not that large, in host count terms, and the maximum practical size per host is similarly not large for the Apple scale, so they are almost certainly sharding objects across hundreds or thousands of clusters. This has nice properties for blast containment.