I operate around $300k monthly mysql infra. We service most schools in the US with a netflix type service. We have burst traffic (at noon EST we x50 traffic).

I operate around $300k monthly mysql infra. We service most schools in the US with a netflix type service. We have burst traffic (at noon EST we x50 traffic). Here is what we learned: 1) RDS/Cloudsql is awesome but too expensive, we can't afford it. The things it provides are not that special with VMs anyway (images, backups, replication). 2) The industry moved to vitess db, but converting to this is too daunting, we are focused on automation and cost reduction instead. 3) Use index based sharding (no math shards % total, no consistent hashing, no drama). When you get a new user, setup where they should be on each shard and keep that on a single table [user_id,video_cluster_id,session_cluster_id,..]. This way moves are easy. Then video_cluster_id is a load balanced host infront of a fleet of replicas. 4) The hardest thing you will run into is write saturation. You can't write fast enough to a disk. Advanced topologies and 3) helps here. Example: [MasterA] <- [Slaves,0..n] <- [MasterB] <- slaveb,0..k]. This way you can push more writes to MasterB, and add replication filters so that slaveb doesn't need to perform all the writes MasterA needs. 5) SSDs. Don't bother with anything else. 6) We would pay a million dollars for this tool: https://docs.percona.com/percona-toolkit/pt-online-schema-change.html https://docs.percona.com/percona-toolkit/pt-online-schema-ch... 7) GTID took away the hard parts of mysql replication. 8) Hardcore mysql experience is disappearing. With RDS/Cloudsql/Postgres less and less developers are going able to patch a production problem.