3 ms·
I worked at LiveJournal and we were sharding there since the late 90s. We ended up with 50 +/- Database pairs and 1 Global Master DB pair serving millions of us
by oceanplexian 2y ago
I worked at LiveJournal and we were sharding there since the late 90s. We ended up with 50 +/- Database pairs and 1 Global Master DB pair serving millions of users in the US (They all started on MySQL and my team moved them over to Percona) We would cut the shards manually when a database would fill up with too many users. We only had to do it once or twice a year, since the only data on the global master was a reference to the user id and the database where your data was stored, it scaled well enough to run the #1 or #2 largest social media site in the US.
Even with early 00s tech you could store more rows in a DB than the population of the USA. When indexed the lookup can be done in milliseconds. If a user database got too hot we would move half the users off it and cut a new user DB, and update their references. Held together with a lot of Perl scripts and duct tape, a far simpler time.