4 ms·

Stupid question: is the because reddits backend is in python? I often hear that PHP + varnish is only limited by the size of the pipe. Can this scale on cloud

• ashray 14y ago
  No, it's probably due to the massive number of reads/writes pumping through their system. I'm pretty sure that their database isn't able to handle what's going on at the moment. Also, if you were to use Varnish as a cache, that wouldn't help you much with write access because every time there is a write (a new post..) you'd have to invalidate the cache. In a high volume scenario such as this (tons of posts coming in..), the page is as good as dynamic, even if you DO cache it for a bit.
  • natrius 14y ago
    Varnish can only really help Reddit with users who aren't logged in since every page is customized. It'd still help in high volume scenarios since Varnish solves the thundering herd problem: only one request (at most) will hit the database for each invalidation.
    • emidln 14y ago
      Vanish can cache snippets of content. They refer to this as Edge Side Includes.
      • natrius 14y ago
        Reddit's customization is far more than snippets. I've used ESIs with Varnish extensively, and I don't think it would help much for authenticated Reddit users.
  • pippy 14y ago
    Thank you, this is the answer that I was looking for
• ubernostrum 14y ago
  reddit's backend could be written in assembly hand-tuned by God Himself, and having half the world hit it at once would still cause bottlenecks at the database.
• Havoc 14y ago
  The problem is that pretty much all of it is dynamically generated. Short of the sidebar & header there is nothing that is really static static. e.g. With all the comments it checks whether that person is on your friend list & colours those differently. Easy enough to do...but it doesn't scale well. They've got a blog explaining the issue(s) somewhere.
• iso-8859-1 14y ago
  Why ask it if you know it's stupid? No, PHP+Varnish can not always scale. You're forgeting the most important important factor in this equation, the database. Varnish is simply for caching and you won't find any large site that doesn't have it. PHP is a scripting language and the runtime performance differences to other common web application languages (i.e. Python, Java) is negligible since you're using caching anyway. Here's what you should worry about if you want scaling: Your database, your data structures and your code quality.
• forensic 14y ago
  It's a stupid question because the particular dialect of assembler this was written in is completely irrelevant for performance considerations. You've been brainwashed by the idiotic pop culture language wars that pretend a question like Ruby vs Python vs whatever has any meaning. These are debates over buzzwords, not reality. All that matters is the database configuration and associated architecture. Not some buzzword drama.
  • xtracto 14y ago
    Wow, news.ycombinator surely has become very offensive and attacking. So much that a person cannot ask questions about IT.
    • forensic 14y ago
      "become" ? 4 years ago, stupid comments were brutally discouraged. People with stupid things to say would be dogpiled by negative comments. It is pop culture that thinks asking stupid questions is a virtue. Most people are just too tactful to tell the truth.
    • pippy 14y ago
      The main issue with his post is he has no idea what he's talking about. He mentions languages neither interpreter is written in and talks about 'buzzwords', while all the products I mentioned are real things.
  • pippy 14y ago
    I'm not sure what you're talking about. Both Python, PHP and Varnish are (mostly) written in higher level languages. Also performance has a massive effect on how and what language a site is written in.
    • forensic 14y ago
      >I'm not sure what you're talking about. My comment is sarcastic and abstract. A literal interpretation will not suffice to understand my meaning. If you want it literally: the language this is written in is irrelevant, because the performance differences between the various languages are negligible next to the performance differences that come about due to the precise design of the website. A poorly engineered website in a "fast" language will still be slower than a well-engineered website in a "slow" language. The actual language is completely irrelevant next to the implementation details. >Also performance has a massive effect on how and what language a site is written in. Not really. Experienced engineers understand that language is basically irrelevant for a situation like this. Whether it is written in Python or something else is irrelevant information. Additionally, newbies tend to focus on "language" and "language wars" and the comparison between languages not for any engineering or technical reason. Newbies do this because languages represent buzzwords around which various communities congregate. The Python vs. Ruby fight is as irrelevant as the Presbyterian vs. Baptist fight. It is the psychological appeal of "identity politics" that motivates people to engage in these meaningless debates about buzzwords.