Scaling does not imply availability. Sharding user data certainly is a good strategy to achieve scalability (i.e., support a large number of users, distributed

Scaling does not imply availability. Sharding user data certainly is a good strategy to achieve scalability (i.e., support a large number of users, distributed across a fleet of commodity hardware), but it doesn't a priori make that data highly available. Let's imagine my personal netflix data (queue, ratings, etc.) live on a lightly loaded (since things are well shareded) DB instance. What happens if the hardware hosting that DB fails? Ok, let's replicate to a slave, and put into place automated failover mechanisms to promote the slave if the master goes down (since with many shards it's really hard to effect a failover manually in a timely fashion). Should the master and slave be in the same datacenter? That after is all is the failure scenario Netflix originally wanted to address -- what happens if your single datacenter goes down (which eventually will happen, no matter how much redundancy you think it may have)? Fine then, separate the master and slave in different DCs (with fat and fast pipes so 2PC or synchronous replication can keep things consistent without impacting latency too much). Now where does the auto-promotion logic live -- the DC with the slave or the master? What happens if my client can reach both DCs, but there are connectivity issues on the links connecting the DCs? Network hardware can fail like any other kind, backhoes accidentally sever buried optics, and routing protocols do take time to converge. P is now rearing its ugly and inevitable head, and you're faced with the C/A decision -- can I see and alter my queue (leading to inconsistency) or is it unavailable? Is this scenario pathological?