3 ms·
Slack does (or did, as of a few years ago) something like this - they shard teams onto a fleet of MySQL servers. See https://www.infoq.com/presentations/slack-i
by geofft 6y ago
Slack does (or did, as of a few years ago) something like this - they shard teams onto a fleet of MySQL servers. See https://www.infoq.com/presentations/slack-infrastructure/ https://www.infoq.com/presentations/slack-infrastructure/ starting around the 10-minute mark and https://www.infoq.com/podcasts/slack-keith-adams/ https://www.infoq.com/podcasts/slack-keith-adams/ starting around 7m15.
If I'm understanding this right, every team gets its own database, which is active-active replicated across two MySQL servers. There's also a separate mapping from teams to MySQL servers (which is itself a separate pair of MySQL servers), and of those two servers, one is primary for each team, distributed roughly evenly. Each MySQL server instance in the fleet is hosting many Slack workspaces, and they can scale that up/down as needed (i.e., presumably the tiny Slack I created years ago and forgot about is hosted on the same pair of MySQL servers as lots of other Slack instances in the same state and my employer's active Slack instance with thousands of people is on a less-crowded pair of MySQL server).
One user-visible effect is that it's possible to have outages that affect some fraction of workspaces. I also suspect this plays into things like rolling out changes over the course of many months - they don't need to do a database migration for everyone at one.
I don't think they use this for security - while yes, a buggy query can only affect one workspace at once, it doesn't sound to me like they do sharding at the PHP layer, it sounds like they're running a fleet of generic PHP instances that have the ability (and authorization) to talk to any workspace's database, not that they're running separate web/PHP instances per customer. But it definitely sounds like they rely on this for scalability/performance.
- dmlittle 6y agoI'm not sure if they were using it at the time Keith gave his presentation but today Slack is using Vitess[1] to handle MySQL clustering. There are a few KubeCon talks about it you're more interested in it. If I remember correctly they're on the tail-end of the Vitess migration and should finish sometime this year. [1] https://vitess.io/ https://vitess.io/