5 ms·

One thing to keep in mind when looking at older MySQL scale-out presentations and war-stories: they're mostly MySQL 5.0-5.6, and a chunk of them are bare metal

One thing to keep in mind when looking at older MySQL scale-out presentations and war-stories: they're mostly MySQL 5.0-5.6, and a chunk of them are bare metal rather than cloud. The problems you'll encounter today -- scaling out in a modern cloud environment on MySQL 8 -- may not line up too well with what folks hit in the past. Ditto for best practices on architecture and ideal max db instance size. I gave a bunch of MySQL conference talks from 2011 through 2017, including several focused on scaling and sharding at social networks. Looking back through them today, a lot of content from the scaling/sharding ones honestly isn't too useful if you're trying to scale up a new MySQL deployment right now.

mooreds 4y ago
What has changed? Is it the performance characteristics of a VM vs bare metal? Is it the characteristics around replication in MySQL 8? Something else? I imagine sharding, read replicas and replication are still a thing, right? Or am I wrong? Is it the particulars or the big strokes that need updating?
- evanelias 4y ago
  It's just a completely different set of scaling pain points. The war stories and anecdotes of the old days just aren't very applicable anymore for newer companies. For example, on bare metal, physically running out of disk space on a hot shard was a relatively common concern. It required a lot of monitoring, automation, and capacity planning to avoid. Whereas on something like AWS Aurora, you mostly just don't worry about it unless/until you're getting into the many-TB range. On bare metal, building out a new data center / region is a huge task. Automating the process was even more complicated. Whereas with large cloud providers, you already have a lot of regions available, and nice APIs to automate things. Hardware of 10-15 years ago had slower IO, less storage, way less RAM, less network bandwidth... and with bare metal you had to deal with multiple generations of hardware in some fashion, often both within a region / data center and between different regions / DCs. This gets especially ugly when you have replicas in each region with very different hardware profiles. Meanwhile in terms of MySQL versions, there's been major strides in replication options/topologies/perf, crash safety, online schema change, automation via MySQL shell, JSON functionality, built-in physical copy/clone, many other things. The third-party tooling ecosystem has also evolved substantially -- monitoring, deployment, HA/failover, proxies, sharding, schema management, backup/restore, etc. Large companies used to all solve this stuff in-house, often in messy home-grown ways... today there's great generic options that are basically usable out-of-the-box.

mooreds 4y ago

What has changed? Is it the performance characteristics of a VM vs bare metal? Is it the characteristics around replication in MySQL 8? Something else? I imagine sharding, read replicas and replication are still a thing, right? Or am I wrong? Is it the particulars or the big strokes that need updating?

evanelias 4y ago
It's just a completely different set of scaling pain points. The war stories and anecdotes of the old days just aren't very applicable anymore for newer companies. For example, on bare metal, physically running out of disk space on a hot shard was a relatively common concern. It required a lot of monitoring, automation, and capacity planning to avoid. Whereas on something like AWS Aurora, you mostly just don't worry about it unless/until you're getting into the many-TB range. On bare metal, building out a new data center / region is a huge task. Automating the process was even more complicated. Whereas with large cloud providers, you already have a lot of regions available, and nice APIs to automate things. Hardware of 10-15 years ago had slower IO, less storage, way less RAM, less network bandwidth... and with bare metal you had to deal with multiple generations of hardware in some fashion, often both within a region / data center and between different regions / DCs. This gets especially ugly when you have replicas in each region with very different hardware profiles. Meanwhile in terms of MySQL versions, there's been major strides in replication options/topologies/perf, crash safety, online schema change, automation via MySQL shell, JSON functionality, built-in physical copy/clone, many other things. The third-party tooling ecosystem has also evolved substantially -- monitoring, deployment, HA/failover, proxies, sharding, schema management, backup/restore, etc. Large companies used to all solve this stuff in-house, often in messy home-grown ways... today there's great generic options that are basically usable out-of-the-box.