2 ms·
from that article it sounds like they are mostly doing "functional partitioning" (moving tables off to other db primary/replica clusters) rather than true shard
by throwusawayus 5y ago
from that article it sounds like they are mostly doing "functional partitioning" (moving tables off to other db primary/replica clusters) rather than true sharding (splitting up tables by ranges of data)
functional partitioning is a band-aid. you do it when your main cluster is exploding but you need to buy time. it ultimately is a very bad thing, because generally your whole site is depenedent on every single functional partition being up. it moves you from 1 single point of failure to N single points of failure!
- wincent 5y agoTowards the end: > In addition to vertical partitioning to move database tables, we also use horizontal partitioning (aka sharding). This allows us to split database tables across multiple clusters, enabling more sustainable growth.
- throwusawayus 5y agoyes i know, the fact that this is a small blurb at the bottom of the article (which is largely about functional partitioning) exactly proves my point
- dikei 5y ago>> functional partitioning is a band-aid. I disagree, functional partitioning is not a band-aid, but an architectural changes that in the end can reap much more benefit than simple data sharding. >> your whole site is dependent on every single functional partition being up. it moves you from 1 single point of failure to N single points of failure! Not necessarily, it can also be that only some parts of your site are dead while others work perfectly fine.
- throwusawayus 5y agotoo idealistic. invariably some team (or usually many teams) don't properly gate some critical-path logic, they depend on some functional partition always being online and then boom much larger blast radius than intended then they fix it in post-mortem but pattern just repeats. i have seen it so many times! used to be much worse in the earlier days of the cloud when VMs would go poof more often