14 ms·

The Evolution of Reddit’s Architecture (2017)

183 points by Jaruzel 8y ago

matkins 8y ago
reddit.com's time spent on each request and uptime are shameful, and they are not getting any better after the redesign.
- Longhanks 8y ago
  The redesign is such a horrible user experience, the day old.reddit.com gets shut down (assuming there is no Desktop program that comes close in quality to the mobile app Apollo) is the day I'm going to stop visiting.
  - rando444 8y ago
    This same sentiment was echoed just as strongly after the last major redesign (when they went from straight text to adding thumbnails and more whitespace). The majority hated it, rebelled against it, said that the change in the way content consumption was focused on images more than text would push out quality content in favor of image macros that can be voted on quickly, that the quality of the people using the site would drop, etc, etc. 10 years later, I still feel all of those concerns were valid and correct, however these changes did not hurt reddit, and in fact the company grew by leaps and bounds. And even now, I still share the same sentiment as you and others, but I think whether or not these changes are actually detrimental to reddit as an entity are yet to be determined.
    - matkins 8y ago
      I've been using reddit for almost a decade. I've found many people on reddit saying that if it wasn't for the redesign (and the mobile apps) they would not be using reddit. It surprises me to see that some people like it; it made me realise this will mark a very, very big demographic shift in the website. Many old users will leave, and many new users will come. Whether that's for good or bad remains to be seen. I think the original spirit of Reddit may end up disappearing, but I don't think that's something that is worth saving, as redditors and their culture are crap. You compare it to having added thumbnails and whitespace, but this is a complete site rewrite, both in the backend and the frontend. Nothing to do with what you mention. Personally I will also quit once they force us to use the redesign because I find it to be slow and, let's say, not "productive" enough.
      - emodendroket 8y ago
        I question whether there is actually a "spirit of Reddit;" it seems like a clearinghouse of wildly different communities with different values and tastes.
    - thecatspaw 8y ago
      > however these changes did not hurt reddit Maybe not in numbers, but the user base changes. Users who value in depth conversations may feel alienated by the changes of reddit, pushed away by such a redesign. I also feel very much alienated by the pushy (almost dark pattern) "use our mobile app" popups on mobile. AMP made that even worse, before that my phone asked me if I wanted to open my reddit app (not the official one), now I get the mobile AMP page with a big "Continue" button, which does not continue to the mobile page, but to the official apps store page. Reddit moved its focus away from discussions, towards beeing an advertisement platform. I've quit it because of that
      - simias 8y ago
        >Maybe not in numbers, but the user base changes. Users who value in depth conversations may feel alienated by the changes of reddit, pushed away by such a redesign. If the end game for reddit is making advertisers happy it probably doesn't matter much. >Reddit moved its focus away from discussions, towards beeing an advertisement platform. It was always an advertisement platform. It's the entire business model. They just needed to be more subtle about it when they were in the bootstraping phase not to drive people away. Now that they managed to reach the critical size where the network effects effectively means that a large portion of their user is captive they can move to monetizing more aggressively.
    - Normal_gaussian 8y ago
      This I think, is the truth of it. Reddit is becoming a lower and lower quality platform, providing less value to users. However each of these moves reduce the barrier to entry (thus increasing user base) or provide value to those who pay them. Often both. Reddit as a business does not want to be a discussion platform. In my opinion it will succeed, but it does so whilst providing mixed value to me, it has destroyed many small communities by amalgamation and then normalising them. Subreddits related to my core interests are useless. Overrun by an endless stream of amateurs and the lack of a consistent user base makes it draining to use and fails to reward involvement at a high skill level (where you need to make repeated connections with others at a high skill level and break the norm). However subreddits related to passing interests are brilliant, the provision of novice level instruction is very common and normally of high quality (due to catering to a constant stream of newcomers and a tendency towards the norm). Reddit is group think.
      - gldalmaso 8y ago
        I can see your point. However how much of that is because of Reddit's decisions in contrast to how the internet itself changed from geek niche to mainstream? Over the years I feel like the entire experience got worse and I might even go so far as to put in question if it isn't a net negative for society. The flood of shallow and biased information has pretty much killed quality in every platform. The signal to noise ratio is abysmal. Hate mongering is rampant world wide fueled by social media. I don't think society is mature enough and ready for this much level of interconnected discourse. Yet here it is. It will be a while until we adapt I believe.
    - swebs 8y ago
      Users aren't going to bounce if there's a CSS change. They are when a page takes more than 5 seconds to load though.
    - hazz99 8y ago
      I don't think it's quite the same - this redesign takes so damn long to load. I'd be okay with it if it was snappy, but I takes over 5 seconds! Each page has mostly text!
    - conradfr 8y ago
      I guess they learned from Digg to not do a massive rollout. That way the new mild users who don't know the old design start invading the site and the day they delete old.reddit they'll only lose a minority of users. IMHO the real bad change the last few years (leaving closing subreddits aside) was the removal of the number of upvote/downvote.
    - wycy 8y ago
      I think the problem is that people always, without exception, hate UI/UX change, and companies know this and have learned to just ignore the objections of their users. Every single change to the Facebook UI/UX, every single change to Windows UI/UX, is met with an enormous backlash from users--"I just want it to look and work like it used to!" Then the next redesign comes around, and lo and behold, now people are attached to the current design that they previously hated, and now hate the new design. Tale as old as time. I've never been like this. I've actually really liked virtually all of the widely panned UI redesigns over the years: iOS 7, every version of Windows, every evolution of Facebook. However, there have been 2 redesigns that even I didn't like: the Digg redesign, and this Reddit redesign. Both are objectively bad. We know what happened because of the former. It remains to be seen what will happen due to the latter.
      - matkins 8y ago
        >I've never been like this. I've actually really liked virtually all of the widely panned UI redesigns over the years: iOS 7, every version of Windows, every evolution of Facebook. Same. And I still hate the new Reddit. Not because of the new design but because of how slow it is and because they have shifted CPU usage from their servers to my computer.
        wycy 8y ago
        Yeah, the new reddit is just objectively bad. It frequently fails to load content for me at all. Terrible user experience.
      - nickjj 8y ago
        > I think the problem is that people always, without exception, hate UI/UX change, and companies know this and have learned to just ignore the objections of their users. I think this is only partly true. Most of the time new designs come with a few side effects like being really slow or buggy, but then over time those things get ironed out. For example, the new gmail UI is a nice improvement but it's buggy. Clicking into an email won't flag it as read if you leave the email quickly so you end up with a bunch of unread emails that you actually read which in turn makes me hate the new UI due to a bug. I'm concerned because this was around way back when they announced the opt in beta and it's still not fixed after forcing it on users.
    - wyldfire 8y ago
      > 10 years later, I still feel all of those concerns were valid and correct, however these changes did not hurt reddit, and in fact the company grew by leaps and bounds. I didn't care much for the thumbnails and changed my preferences to disable them. Even though I agree that image macros/memes diminished reddit's quality I must confess to being a party to consuming (and upvoting) these posts. HN fills this gap well, IMO. IMO this new redesign is different from that one. That one was much more cosmetic. Impact-wise, this is somewhere between when they added support for comments and that thumbnails change. The pace of the page rendering / reaction is significantly different from the baseline and it changes how we consume reddit.
  - craigsmansion 8y ago
    Reddit could really do with a NNTP gateway. It's basically a centralised usenet with voties anyway. Sure, you'd probably lose the option to vote on posts, but for the subreddits I frequent I don't really care. It would also let users use a client of their choosing and never having to worry about ghastly site updates again. Maybe I'll eventually get around to it, but for now old.reddit will have to do.
- Theizestooke 8y ago
  reddit video is particularly dreadful. I usually skip the link if it's reddit video.
  - adrianN 8y ago
    I have a Greasemonkey script that hides all v.reddit.com posts.
Fri31Aug 8y ago
Reddit is the only major site I know which is regularly (well not so often, but often enough for me to realize) unplanned, and planned, downtime. Makes me wonder whether their architecture is really something to follow.
- throwaway427 8y ago
  I've always thought some sort of straightforward partitioning strategy, even the simple one from Building Scalable Web Sites[1], or the Instagram model, would fit pretty nicely over most over multiple Reddit product dimensions -- partition by subreddit (with special instances for exceptionally busy reddits), partition by domain, etc. [1] Hard to believe that book came out 12 years ago edit: s/shard/partition
- saosebastiao 8y ago
  Without even looking at architecture, they went from lisp to python to node. While I certainly wouldn't have chosen any of those languages, it seems like every step they took was a step backwards. What's next, tcl/tk?
  - vladadaba 8y ago
    Why is that and which language would you choose? I am asking because I want to get into web development and started learning React + NodeJS recently.
    - edoceo 8y ago
      Keep learning those two, they are valuable in the job market. Don't lose core focus on language (js) vs framework/tooling. Knowing the core language is the adaptable skill.
    - fianhe 8y ago
      React and Node are well engineered and in demand, plus you can even use it to build mobile app with React Native, which is in demand too. Don't listen to advices from anonymous random guy from the internet, doubt even me.
pjc50 8y ago
(Based on the transcript, thanks for providing that, I don't have time for videos) The summary would seem to be that the first generation of developers built a monolith. A reasonably well scaling monolith, but only up to a point. Newer developers fear the blob and prefer Node to Python, so new development has been done in a cluster of menhirs of node around the python monolith. The monolith suffered from lock contention because it was inadequately sharded across subreddits. They have now addressed this. (Not mentioned: the redesign which shifts a large amount of CPU usage onto the client)
pg_bot 8y ago
If I were to design reddit I would have enforced the following rule to handle their scaling problems. You cannot touch the database when you are reading the homepage. Instead every minute we generate the canonical list of ~500 links that is "reddit". Cache the html for that and throw it on a CDN. The "next" and "previous" timestamped links should reference when it was first loaded so you can continue paging and avoid the duplicate link problem. You can cheat for logged in users by loading in their data after page load with JS, but the site would never fail entirely if it were overloaded. Votes should never affect another record, they should always create a new record in your datastore to avoid locking problems.
- intended 8y ago
  If I understand you correctly, there’s a new row entry for every vote ? I
  - extra88 8y ago
    I would absolutely expect every vote to be recorded in a row with at least up/down, by whom, and when. You can browse your own votes on posts, I don't know if there's a way to browse votes on comments. https://www.reddit.com/user/USERNAME/upvoted/ https://www.reddit.com/user/USERNAME/upvoted/ https://www.reddit.com/user/USERNAME/downvoted/ https://www.reddit.com/user/USERNAME/downvoted/
    - threeseed 8y ago
      You realise that Reddit is over 10 years old. There would have been trillions of votes since then. What database are you thinking that could store all of those and then I assume aggregate them ? Kind of curious about this proposed architecture.
      - extra88 8y ago
        I would expect vote sums to be stored separately, you don't need to query a table containing all votes on every page load any more than a bank needs to query all transactions to show your account balance.
TekMol 8y ago
The Reddit redesign must be the worst redesign in the history of redesigns. It's super slow to load. It's slow to use (Scroll is sluggish. It loads giant images and puts huge autoplaying videos in your face. Comment function now starts an ugly html wysiwyg editor). It misses features (outgoing links in the sidebars are gone). It takes away screen estate by having a fixed element on top of the screen that stays there when you scroll down. It's buggy (often weird overlaps between elements happen when you resize the screen). It's confusing. When I click on a post I get a view that shows the post in an overlay of the page I was looking at. When I reload the page I suddenly get a completely different view with only the post. It's almost unusable on my iPad (where I have to beg for the desktop version every time I open it because the mobile version is even worse). It seems like they are still A/B testing it. Because sometimes I get the redesign and sometimes the original design. Funny enough, I can see how the redesign might increase user engagement. The endless scroll with content right in your face is kind of addicting. It pulls you in a state of "I hate this. And I hate myself for using it. But I can't stop. I will just scroll down one post further.". So the experience is worse but the KPIs might be better. I bet this is how booking.com became the hell that it is. I would love to see how the numbers of those tests look like.
- invalidusernam3 8y ago
  This thread in /r/dataisbeautiful seemed to indicate most people hate the new design and avoid it (old.reddit.com) - https://old.reddit.com/r/dataisbeautiful/comments/9e4gvn/reddits_opinion_on_the_redesign_who_loves_it_and/ https://old.reddit.com/r/dataisbeautiful/comments/9e4gvn/red...
  - wingerlang 8y ago
    7k data point here basically means 7k people that care enough / dislike the redesign enough to be involved in the poll to begin with. How many users each day doesn't give a hoot about the redesign? 100's of thousands?
    - user812 8y ago
      I'm a mod of a small subreddit and during the last 3 months around 10-15% of our visitors used the redesign, 20% browse on mobile, 30% use an App, and the rest use the old design. Engagement is highest in the Apps.
justtopost 8y ago
The redesign effectively ended my use of this site completely. It is so slow and agressively user hostile, it blows my mind anyonw would want to 'learn' from it in any way but avoidance.
- vezycash 8y ago
  Old.reddit.com Bookmark the version of your favorite sub-reddits for mobile use. Manually edit https:www.reddit.com/post to old.reddit.com/post OR Use a chrome or Firefox extension on PC to automatically redirect reddit URLs to old.reddit.com version. Problem solved for now
  - endisukaj 8y ago
    You can also opt out in the user settings if you have an account.
rcavezza 8y ago
I just watched the entire video and it was very informative. Fwiw most of the comments on here did not watch the video and seem a little uninformed. The node apis were built by a small group of front end developers and most of the pulled out services they're building are in python . There were also a lot about message queues, cacheing, and ops related things in the talk. I definitely recommend watching the entire video if you're interested in high traffic architectures.
- sethammons 8y ago
  While those basic strategies are technically good for scaling, I think the other commenters have a very valid point: it seems a lot of folks say, "no, this strategy is not something that appears to be working because the site is slow, buggy, and, oh, did I mention slow." I think many folks consider scaling and fast a success and scaling and slow/buggy a failure. Who wants to emulate failure?
overcast 8y ago
The great irony here, is that Reddit is making the same foolish mistake Digg did with a redesign, costing them their entire userbase, and propelling Reddit.
- ProAm 8y ago
  I always say Reddit is a combination of terrible content from Facebook, the comments from Twitter and the redesigns from Snap and Digg. I've actually gone back to Digg for interesting stuff on the web. Interesting content, no comments and it usually has about 20-40 articles a day so I dont feel the need to check it every 5 minutes.
  - jplayer01 8y ago
    > combination of terrible content from Facebook, the comments from Twitter This just shows how little you actually use or understand Reddit. I'm not sure what I'd do without subs like r/economics, r/badeconomics, r/math, r/learnmath, r/programming, r/<insert programming language here>, any number of friendly and welcoming fetish communities (that aren't just about posting porn), r/fitness, r/keto, r/running, r/weightlifting, etc. Reddit is just an unrivaled, incredible place where information is gathered and opinions can be shared and support given.
    - nickthemagicman 8y ago
      Agreed. I've learned incredible amount from both Reddit and Hacker News. It's all about proper content selection.
      - overcast 8y ago
        Was that from actually using Reddit daily though? Or just Googling, and using Reddit as a result?
        jplayer01 8y ago
        How does that matter? If the information is gathered on Reddit in an accessible, helpful and easy to understand way, it doesn't matter whether you get there from hacker news, SO, google, bing or through browsing Reddit. It's still on Reddit.
OnlyRepliesToBS 8y ago
pls stop
justinhj 8y ago
I hate using Reddit on mobile. I’m generally reading on a bus or train in Safari and it will virtually force me to use the app. 90% of those times it will open the store instead and maybe want to update the app, or I’m not logged in so I will have to take a few minutes to reset the password, even though I’m logged into the web app. If the app does load it’s slow and buggy. I wish I could make the mobile web page the default.
- slazaro 8y ago
  Have you tried 3rd party apps? On Android I use Relay and it works great, I assume on iOS there are similarly good apps.
  - justinhj 8y ago
    Thanks for the tip, I'll investigate

183 points by Jaruzel 8y ago

reddit.com's time spent on each request and uptime are shameful, and they are not getting any better after the redesign.

Longhanks 8y ago
The redesign is such a horrible user experience, the day old.reddit.com gets shut down (assuming there is no Desktop program that comes close in quality to the mobile app Apollo) is the day I'm going to stop visiting.
- rando444 8y ago
  This same sentiment was echoed just as strongly after the last major redesign (when they went from straight text to adding thumbnails and more whitespace). The majority hated it, rebelled against it, said that the change in the way content consumption was focused on images more than text would push out quality content in favor of image macros that can be voted on quickly, that the quality of the people using the site would drop, etc, etc. 10 years later, I still feel all of those concerns were valid and correct, however these changes did not hurt reddit, and in fact the company grew by leaps and bounds. And even now, I still share the same sentiment as you and others, but I think whether or not these changes are actually detrimental to reddit as an entity are yet to be determined.
  - matkins 8y ago
    I've been using reddit for almost a decade. I've found many people on reddit saying that if it wasn't for the redesign (and the mobile apps) they would not be using reddit. It surprises me to see that some people like it; it made me realise this will mark a very, very big demographic shift in the website. Many old users will leave, and many new users will come. Whether that's for good or bad remains to be seen. I think the original spirit of Reddit may end up disappearing, but I don't think that's something that is worth saving, as redditors and their culture are crap. You compare it to having added thumbnails and whitespace, but this is a complete site rewrite, both in the backend and the frontend. Nothing to do with what you mention. Personally I will also quit once they force us to use the redesign because I find it to be slow and, let's say, not "productive" enough.
    - emodendroket 8y ago
      I question whether there is actually a "spirit of Reddit;" it seems like a clearinghouse of wildly different communities with different values and tastes.
  - thecatspaw 8y ago
    > however these changes did not hurt reddit Maybe not in numbers, but the user base changes. Users who value in depth conversations may feel alienated by the changes of reddit, pushed away by such a redesign. I also feel very much alienated by the pushy (almost dark pattern) "use our mobile app" popups on mobile. AMP made that even worse, before that my phone asked me if I wanted to open my reddit app (not the official one), now I get the mobile AMP page with a big "Continue" button, which does not continue to the mobile page, but to the official apps store page. Reddit moved its focus away from discussions, towards beeing an advertisement platform. I've quit it because of that
    - simias 8y ago
      >Maybe not in numbers, but the user base changes. Users who value in depth conversations may feel alienated by the changes of reddit, pushed away by such a redesign. If the end game for reddit is making advertisers happy it probably doesn't matter much. >Reddit moved its focus away from discussions, towards beeing an advertisement platform. It was always an advertisement platform. It's the entire business model. They just needed to be more subtle about it when they were in the bootstraping phase not to drive people away. Now that they managed to reach the critical size where the network effects effectively means that a large portion of their user is captive they can move to monetizing more aggressively.
  - Normal_gaussian 8y ago
    This I think, is the truth of it. Reddit is becoming a lower and lower quality platform, providing less value to users. However each of these moves reduce the barrier to entry (thus increasing user base) or provide value to those who pay them. Often both. Reddit as a business does not want to be a discussion platform. In my opinion it will succeed, but it does so whilst providing mixed value to me, it has destroyed many small communities by amalgamation and then normalising them. Subreddits related to my core interests are useless. Overrun by an endless stream of amateurs and the lack of a consistent user base makes it draining to use and fails to reward involvement at a high skill level (where you need to make repeated connections with others at a high skill level and break the norm). However subreddits related to passing interests are brilliant, the provision of novice level instruction is very common and normally of high quality (due to catering to a constant stream of newcomers and a tendency towards the norm). Reddit is group think.
    - gldalmaso 8y ago
      I can see your point. However how much of that is because of Reddit's decisions in contrast to how the internet itself changed from geek niche to mainstream? Over the years I feel like the entire experience got worse and I might even go so far as to put in question if it isn't a net negative for society. The flood of shallow and biased information has pretty much killed quality in every platform. The signal to noise ratio is abysmal. Hate mongering is rampant world wide fueled by social media. I don't think society is mature enough and ready for this much level of interconnected discourse. Yet here it is. It will be a while until we adapt I believe.
  - swebs 8y ago
    Users aren't going to bounce if there's a CSS change. They are when a page takes more than 5 seconds to load though.
  - hazz99 8y ago
    I don't think it's quite the same - this redesign takes so damn long to load. I'd be okay with it if it was snappy, but I takes over 5 seconds! Each page has mostly text!
  - conradfr 8y ago
    I guess they learned from Digg to not do a massive rollout. That way the new mild users who don't know the old design start invading the site and the day they delete old.reddit they'll only lose a minority of users. IMHO the real bad change the last few years (leaving closing subreddits aside) was the removal of the number of upvote/downvote.
  - wycy 8y ago
    I think the problem is that people always, without exception, hate UI/UX change, and companies know this and have learned to just ignore the objections of their users. Every single change to the Facebook UI/UX, every single change to Windows UI/UX, is met with an enormous backlash from users--"I just want it to look and work like it used to!" Then the next redesign comes around, and lo and behold, now people are attached to the current design that they previously hated, and now hate the new design. Tale as old as time. I've never been like this. I've actually really liked virtually all of the widely panned UI redesigns over the years: iOS 7, every version of Windows, every evolution of Facebook. However, there have been 2 redesigns that even I didn't like: the Digg redesign, and this Reddit redesign. Both are objectively bad. We know what happened because of the former. It remains to be seen what will happen due to the latter.
    - matkins 8y ago
      >I've never been like this. I've actually really liked virtually all of the widely panned UI redesigns over the years: iOS 7, every version of Windows, every evolution of Facebook. Same. And I still hate the new Reddit. Not because of the new design but because of how slow it is and because they have shifted CPU usage from their servers to my computer.
      - wycy 8y ago
        Yeah, the new reddit is just objectively bad. It frequently fails to load content for me at all. Terrible user experience.
    - nickjj 8y ago
      > I think the problem is that people always, without exception, hate UI/UX change, and companies know this and have learned to just ignore the objections of their users. I think this is only partly true. Most of the time new designs come with a few side effects like being really slow or buggy, but then over time those things get ironed out. For example, the new gmail UI is a nice improvement but it's buggy. Clicking into an email won't flag it as read if you leave the email quickly so you end up with a bunch of unread emails that you actually read which in turn makes me hate the new UI due to a bug. I'm concerned because this was around way back when they announced the opt in beta and it's still not fixed after forcing it on users.
  - wyldfire 8y ago
    > 10 years later, I still feel all of those concerns were valid and correct, however these changes did not hurt reddit, and in fact the company grew by leaps and bounds. I didn't care much for the thumbnails and changed my preferences to disable them. Even though I agree that image macros/memes diminished reddit's quality I must confess to being a party to consuming (and upvoting) these posts. HN fills this gap well, IMO. IMO this new redesign is different from that one. That one was much more cosmetic. Impact-wise, this is somewhere between when they added support for comments and that thumbnails change. The pace of the page rendering / reaction is significantly different from the baseline and it changes how we consume reddit.
- craigsmansion 8y ago
  Reddit could really do with a NNTP gateway. It's basically a centralised usenet with voties anyway. Sure, you'd probably lose the option to vote on posts, but for the subreddits I frequent I don't really care. It would also let users use a client of their choosing and never having to worry about ghastly site updates again. Maybe I'll eventually get around to it, but for now old.reddit will have to do.
Theizestooke 8y ago
reddit video is particularly dreadful. I usually skip the link if it's reddit video.
- adrianN 8y ago
  I have a Greasemonkey script that hides all v.reddit.com posts.

Fri31Aug 8y ago

Reddit is the only major site I know which is regularly (well not so often, but often enough for me to realize) unplanned, and planned, downtime. Makes me wonder whether their architecture is really something to follow.

throwaway427 8y ago
I've always thought some sort of straightforward partitioning strategy, even the simple one from Building Scalable Web Sites[1], or the Instagram model, would fit pretty nicely over most over multiple Reddit product dimensions -- partition by subreddit (with special instances for exceptionally busy reddits), partition by domain, etc. [1] Hard to believe that book came out 12 years ago edit: s/shard/partition
saosebastiao 8y ago
Without even looking at architecture, they went from lisp to python to node. While I certainly wouldn't have chosen any of those languages, it seems like every step they took was a step backwards. What's next, tcl/tk?
- vladadaba 8y ago
  Why is that and which language would you choose? I am asking because I want to get into web development and started learning React + NodeJS recently.
  - edoceo 8y ago
    Keep learning those two, they are valuable in the job market. Don't lose core focus on language (js) vs framework/tooling. Knowing the core language is the adaptable skill.
  - fianhe 8y ago
    React and Node are well engineered and in demand, plus you can even use it to build mobile app with React Native, which is in demand too. Don't listen to advices from anonymous random guy from the internet, doubt even me.

pjc50 8y ago

(Based on the transcript, thanks for providing that, I don't have time for videos) The summary would seem to be that the first generation of developers built a monolith. A reasonably well scaling monolith, but only up to a point. Newer developers fear the blob and prefer Node to Python, so new development has been done in a cluster of menhirs of node around the python monolith. The monolith suffered from lock contention because it was inadequately sharded across subreddits. They have now addressed this. (Not mentioned: the redesign which shifts a large amount of CPU usage onto the client)

pg_bot 8y ago

If I were to design reddit I would have enforced the following rule to handle their scaling problems. You cannot touch the database when you are reading the homepage. Instead every minute we generate the canonical list of ~500 links that is "reddit". Cache the html for that and throw it on a CDN. The "next" and "previous" timestamped links should reference when it was first loaded so you can continue paging and avoid the duplicate link problem. You can cheat for logged in users by loading in their data after page load with JS, but the site would never fail entirely if it were overloaded. Votes should never affect another record, they should always create a new record in your datastore to avoid locking problems.

intended 8y ago
If I understand you correctly, there’s a new row entry for every vote ? I
- extra88 8y ago
  I would absolutely expect every vote to be recorded in a row with at least up/down, by whom, and when. You can browse your own votes on posts, I don't know if there's a way to browse votes on comments. https://www.reddit.com/user/USERNAME/upvoted/ https://www.reddit.com/user/USERNAME/upvoted/ https://www.reddit.com/user/USERNAME/downvoted/ https://www.reddit.com/user/USERNAME/downvoted/
  - threeseed 8y ago
    You realise that Reddit is over 10 years old. There would have been trillions of votes since then. What database are you thinking that could store all of those and then I assume aggregate them ? Kind of curious about this proposed architecture.
    - extra88 8y ago
      I would expect vote sums to be stored separately, you don't need to query a table containing all votes on every page load any more than a bank needs to query all transactions to show your account balance.

TekMol 8y ago

The Reddit redesign must be the worst redesign in the history of redesigns. It's super slow to load. It's slow to use (Scroll is sluggish. It loads giant images and puts huge autoplaying videos in your face. Comment function now starts an ugly html wysiwyg editor). It misses features (outgoing links in the sidebars are gone). It takes away screen estate by having a fixed element on top of the screen that stays there when you scroll down. It's buggy (often weird overlaps between elements happen when you resize the screen). It's confusing. When I click on a post I get a view that shows the post in an overlay of the page I was looking at. When I reload the page I suddenly get a completely different view with only the post. It's almost unusable on my iPad (where I have to beg for the desktop version every time I open it because the mobile version is even worse). It seems like they are still A/B testing it. Because sometimes I get the redesign and sometimes the original design. Funny enough, I can see how the redesign might increase user engagement. The endless scroll with content right in your face is kind of addicting. It pulls you in a state of "I hate this. And I hate myself for using it. But I can't stop. I will just scroll down one post further.". So the experience is worse but the KPIs might be better. I bet this is how booking.com became the hell that it is. I would love to see how the numbers of those tests look like.

invalidusernam3 8y ago
This thread in /r/dataisbeautiful seemed to indicate most people hate the new design and avoid it (old.reddit.com) - https://old.reddit.com/r/dataisbeautiful/comments/9e4gvn/reddits_opinion_on_the_redesign_who_loves_it_and/ https://old.reddit.com/r/dataisbeautiful/comments/9e4gvn/red...
- wingerlang 8y ago
  7k data point here basically means 7k people that care enough / dislike the redesign enough to be involved in the poll to begin with. How many users each day doesn't give a hoot about the redesign? 100's of thousands?
  - user812 8y ago
    I'm a mod of a small subreddit and during the last 3 months around 10-15% of our visitors used the redesign, 20% browse on mobile, 30% use an App, and the rest use the old design. Engagement is highest in the Apps.

justtopost 8y ago

The redesign effectively ended my use of this site completely. It is so slow and agressively user hostile, it blows my mind anyonw would want to 'learn' from it in any way but avoidance.

vezycash 8y ago
Old.reddit.com Bookmark the version of your favorite sub-reddits for mobile use. Manually edit https:www.reddit.com/post to old.reddit.com/post OR Use a chrome or Firefox extension on PC to automatically redirect reddit URLs to old.reddit.com version. Problem solved for now
- endisukaj 8y ago
  You can also opt out in the user settings if you have an account.

rcavezza 8y ago

I just watched the entire video and it was very informative. Fwiw most of the comments on here did not watch the video and seem a little uninformed. The node apis were built by a small group of front end developers and most of the pulled out services they're building are in python . There were also a lot about message queues, cacheing, and ops related things in the talk. I definitely recommend watching the entire video if you're interested in high traffic architectures.

sethammons 8y ago
While those basic strategies are technically good for scaling, I think the other commenters have a very valid point: it seems a lot of folks say, "no, this strategy is not something that appears to be working because the site is slow, buggy, and, oh, did I mention slow." I think many folks consider scaling and fast a success and scaling and slow/buggy a failure. Who wants to emulate failure?

overcast 8y ago

The great irony here, is that Reddit is making the same foolish mistake Digg did with a redesign, costing them their entire userbase, and propelling Reddit.

ProAm 8y ago
I always say Reddit is a combination of terrible content from Facebook, the comments from Twitter and the redesigns from Snap and Digg. I've actually gone back to Digg for interesting stuff on the web. Interesting content, no comments and it usually has about 20-40 articles a day so I dont feel the need to check it every 5 minutes.
- jplayer01 8y ago
  > combination of terrible content from Facebook, the comments from Twitter This just shows how little you actually use or understand Reddit. I'm not sure what I'd do without subs like r/economics, r/badeconomics, r/math, r/learnmath, r/programming, r/<insert programming language here>, any number of friendly and welcoming fetish communities (that aren't just about posting porn), r/fitness, r/keto, r/running, r/weightlifting, etc. Reddit is just an unrivaled, incredible place where information is gathered and opinions can be shared and support given.
  - nickthemagicman 8y ago
    Agreed. I've learned incredible amount from both Reddit and Hacker News. It's all about proper content selection.
    - overcast 8y ago
      Was that from actually using Reddit daily though? Or just Googling, and using Reddit as a result?
      - jplayer01 8y ago
        How does that matter? If the information is gathered on Reddit in an accessible, helpful and easy to understand way, it doesn't matter whether you get there from hacker news, SO, google, bing or through browsing Reddit. It's still on Reddit.

OnlyRepliesToBS 8y ago

pls stop

justinhj 8y ago

I hate using Reddit on mobile. I’m generally reading on a bus or train in Safari and it will virtually force me to use the app. 90% of those times it will open the store instead and maybe want to update the app, or I’m not logged in so I will have to take a few minutes to reset the password, even though I’m logged into the web app. If the app does load it’s slow and buggy. I wish I could make the mobile web page the default.

slazaro 8y ago
Have you tried 3rd party apps? On Android I use Relay and it works great, I assume on iOS there are similarly good apps.
- justinhj 8y ago
  Thanks for the tip, I'll investigate