3 ms·

Shard the databases by subreddit and you will be able to horizontally scale the load. Use JS to post to a separate subdomain for each service. I.E everything

Shard the databases by subreddit and you will be able to horizontally scale the load. Use JS to post to a separate subdomain for each service. I.E everything under www.reddit.com/r/funny points to funny.reddit.com which can live as its own independent service. Internally I would treat reddit as a group of individual high traffic sites instead of one massively trafficked site. Then you have another service that queries each site to build the main page. this is terrifying to read, in that this is exactly what the media companies did for the last decade to try, and totally fail, to compete with facebook. having a unified platform is a competitive advantage. also your "sharding and subdomains" trick is straight out of 2010 and a mix of irrelevant (everything's sharded now) and anachronistic (h2 is everywhere now). also-also, you clearly have missed the best use case of reddit, which is to have unsub'd from all/most of the "default" reddits and replaced them with your own customized set. you are optimizing for everyone reading the same thing (newspaper homepage) not everyone reading their custom thing (social network). that has been a proven losing strategy for a solid decade now, you could argue 2.

pg_bot 8y ago
Everyone is reading the same thing on reddit, just different combinations of the same thing. Reddit is just a combination of multiple newspapers masquerading as a single custom newspaper. No reason why the internals should leak to what is shown to the user. You could keep the exact same user experience that you have today and guarantee that the site never falls over. You need to make the distinction between UX and Operations. Everything the user sees still lives under the same structure as today but internally it is structured so that you don't overload yourself and if you do fail, those failures are isolated and don't take down the rest of the site.
- cagenut 8y ago
  Everyone is reading the same thing on reddit, just different combinations of the same thing. this is categorically untrue, you don't seem to understand reddit. Reddit is just a combination of multiple newspapers masquerading as a single custom newspaper. that is exactly the anachronistic thinking that leads you down this path. its movable type vs wordpress all over again. wordpress won. the engineering constraint of pre-calculating caches is worse, in the long run, than the occasional outage for casual/non-logged-in users. its optimizing for the wrong audience, it is a long slow path to failure.

pg_bot 8y ago

Everyone is reading the same thing on reddit, just different combinations of the same thing. Reddit is just a combination of multiple newspapers masquerading as a single custom newspaper. No reason why the internals should leak to what is shown to the user. You could keep the exact same user experience that you have today and guarantee that the site never falls over. You need to make the distinction between UX and Operations. Everything the user sees still lives under the same structure as today but internally it is structured so that you don't overload yourself and if you do fail, those failures are isolated and don't take down the rest of the site.

cagenut 8y ago
Everyone is reading the same thing on reddit, just different combinations of the same thing. this is categorically untrue, you don't seem to understand reddit. Reddit is just a combination of multiple newspapers masquerading as a single custom newspaper. that is exactly the anachronistic thinking that leads you down this path. its movable type vs wordpress all over again. wordpress won. the engineering constraint of pre-calculating caches is worse, in the long run, than the occasional outage for casual/non-logged-in users. its optimizing for the wrong audience, it is a long slow path to failure.