3 ms·

Funny to see people quick to make fun of a project for not being able to instantly handle a spike in requests, which most of their own apps wouldn't either! Re

by BowBun 3y ago

Funny to see people quick to make fun of a project for not being able to instantly handle a spike in requests, which most of their own apps wouldn't either! Reddit has been around nearly 20 years to harden their infrastructure. Relax and give them a minute. Their page loads for me now that I've typed this out, anyway.

paxys 3y ago
Who is making fun of it?
riceart 3y ago
Let’s be real here this system is flawed at its core though. This can’t even handle the load of some moderately popular PHP forums with simple deployment or even this site nothing took decades to harden. And this is due to gross flaws in architecture, IMNSHO. Comparing it to Reddit is silly. And yes I’m sure there are more than a few here that can honestly say their servers could handle this load - it isn’t much.
- sudosysgen 3y ago
  Care to point out the core flaws? As far as I'm concerned there aren't any core issues. There are a lot of queries to a single DB which is one the same server, but there's nothing preventing you from running the DB on another server and sharding it if need be. I wouldn't call that a core architectural issue because the way you set up your DB is not core to the architecture.
  - riceart 3y ago
    It’s a federated system without any of the affordances that would make it usable (or particularly interesting to me). Literally load balancing by going to a long list of alternate instances. The sites own documentation just says, find one that works, and if that shits the bed find another. This is a UX agreeable to a very tiny and idiosyncratic group (as evidenced by Mastodon once the Elon is evil hype died. Yes of course in theory you can beef up any instance, and shard, and add a caching and queueing layer while you’re at it. And tada you’ve just redid Reddit.
    - sudosysgen 3y ago
      Sharding, caching and queuing doesn't break federation. That's not a core flaw. As the ecosystem grows, a shortlist of popular, robust, federated instances will crop up for people who just don't want to bother. Again, if you have any core flaws, I'm all ears.
      - riceart 3y ago
        > Sharding, caching and queuing doesn't break federation. That's not a core flaw. I didn’t say it did, but it doesn’t enhance it either. > As the ecosystem grows, a shortlist of popular, robust, federated instances will crop up How short is a shortlist? When does that mean just new Reddit? I suppose I would be more enthused if at the least the basic design eased standing up high traffic (or let’s be honest even mild traffic) instances. The performance story right now is: it’s written in rust - which is not nothing but it would be more interesting if supporting even a moderate amount of traffic on minimal hardware was an architectural priority. The flaw as I see it is that this is just not a design goal. https://github.com/LemmyNet/lemmy/issues/2877 https://github.com/LemmyNet/lemmy/issues/2877 https://github.com/LemmyNet/lemmy/issues/2910 https://github.com/LemmyNet/lemmy/issues/2910 You can argue that things can be improved but initially well engineered systems help to enhance initial mindshare.
        sudosysgen 3y ago
        Even just two-three large and popular instances is enough to make a Reddit impossible, so that's how short it can get. The performance story isn't just that the backend is written in Rust. It's also that the front is very lightweight (80kB), that the architecture is horizontally scalable, etc... I don't see in any of those links a core flaw, a problem in the initial engineering that makes it impossible for the architecture to scale. All I see are some poorly written queries, for which the main devs made a root cause analysis and described how it can be fixed. Is that evidence that the inherent design isn't capable of adequate performance? No, it isn't. It's evidence that there are some small performance gotchas that can be fixed easily, and this is normal for an ambitious project. But even as it is, yes, it can handle mild traffic. lemmy.ml runs on potato hardware, hexbear.net is an example of an instance which has ~3'000 comments a day, which is roughly the scale of this website, and it runs fine on a single dedicated server.
AviationAtom 3y ago
They talk a little about the state of the project and it's ability to scale here: https://beehaw.org/post/438094 https://beehaw.org/post/438094

by BowBun 3y ago

paxys 3y ago

Who is making fun of it?

riceart 3y ago

Let’s be real here this system is flawed at its core though. This can’t even handle the load of some moderately popular PHP forums with simple deployment or even this site nothing took decades to harden. And this is due to gross flaws in architecture, IMNSHO. Comparing it to Reddit is silly. And yes I’m sure there are more than a few here that can honestly say their servers could handle this load - it isn’t much.

sudosysgen 3y ago
Care to point out the core flaws? As far as I'm concerned there aren't any core issues. There are a lot of queries to a single DB which is one the same server, but there's nothing preventing you from running the DB on another server and sharding it if need be. I wouldn't call that a core architectural issue because the way you set up your DB is not core to the architecture.
- riceart 3y ago
  It’s a federated system without any of the affordances that would make it usable (or particularly interesting to me). Literally load balancing by going to a long list of alternate instances. The sites own documentation just says, find one that works, and if that shits the bed find another. This is a UX agreeable to a very tiny and idiosyncratic group (as evidenced by Mastodon once the Elon is evil hype died. Yes of course in theory you can beef up any instance, and shard, and add a caching and queueing layer while you’re at it. And tada you’ve just redid Reddit.
  - sudosysgen 3y ago
    Sharding, caching and queuing doesn't break federation. That's not a core flaw. As the ecosystem grows, a shortlist of popular, robust, federated instances will crop up for people who just don't want to bother. Again, if you have any core flaws, I'm all ears.
    - riceart 3y ago
      > Sharding, caching and queuing doesn't break federation. That's not a core flaw. I didn’t say it did, but it doesn’t enhance it either. > As the ecosystem grows, a shortlist of popular, robust, federated instances will crop up How short is a shortlist? When does that mean just new Reddit? I suppose I would be more enthused if at the least the basic design eased standing up high traffic (or let’s be honest even mild traffic) instances. The performance story right now is: it’s written in rust - which is not nothing but it would be more interesting if supporting even a moderate amount of traffic on minimal hardware was an architectural priority. The flaw as I see it is that this is just not a design goal. https://github.com/LemmyNet/lemmy/issues/2877 https://github.com/LemmyNet/lemmy/issues/2877 https://github.com/LemmyNet/lemmy/issues/2910 https://github.com/LemmyNet/lemmy/issues/2910 You can argue that things can be improved but initially well engineered systems help to enhance initial mindshare.
      - sudosysgen 3y ago
        Even just two-three large and popular instances is enough to make a Reddit impossible, so that's how short it can get. The performance story isn't just that the backend is written in Rust. It's also that the front is very lightweight (80kB), that the architecture is horizontally scalable, etc... I don't see in any of those links a core flaw, a problem in the initial engineering that makes it impossible for the architecture to scale. All I see are some poorly written queries, for which the main devs made a root cause analysis and described how it can be fixed. Is that evidence that the inherent design isn't capable of adequate performance? No, it isn't. It's evidence that there are some small performance gotchas that can be fixed easily, and this is normal for an ambitious project. But even as it is, yes, it can handle mild traffic. lemmy.ml runs on potato hardware, hexbear.net is an example of an instance which has ~3'000 comments a day, which is roughly the scale of this website, and it runs fine on a single dedicated server.

AviationAtom 3y ago

They talk a little about the state of the project and it's ability to scale here: https://beehaw.org/post/438094 https://beehaw.org/post/438094