3 ms·

We're more decentralized in fedi but we're also not really consistent either. Which I think is the number one gripe for users who manage to get into the fedi.

We're more decentralized in fedi but we're also not really consistent either. Which I think is the number one gripe for users who manage to get into the fedi. I don't mind, I still think it's a huge leap forward, but it's important to set realistic expectations.

torben-friis 1y ago
What do you mean by consistent? (Never used the fediverse, so zero context here).
- skybrian 1y ago
  Not who you're replying to, but one issue is that different users will sometimes see a different subset of replies to a post, depending on what the server they're using has copied from other servers.
  - alethic 1y ago
    This isn't correct. Mastodon merged fetch-all-replies in March. https://github.com/mastodon/mastodon/pull/32615 https://github.com/mastodon/mastodon/pull/32615 The only difference in visible replies is in the moderation choices of the server the post is viewed from.
    - skybrian 1y ago
      Good to know. Thanks for the update!
    - danabramov 1y ago
      How does it work? In ATProto, there is no need to do this on-demand because the data is already there in the AppView. When you want to serve a page of replies, you read them from the database and serve them. There is no distributed fetching involved, no need to hit someone else's servers, no need to coalesce them or worry about limiting fetches, etc. This is why it works fine for threads without thousands of replies and hundreds of nesting levels. It can also be paginated on the server. If you don't have this information on your server, how can you gracefully fetch thousands of replies from different servers and present a cohesive picture during a single request? I'm sure this PR does an attempt at that but I'm not sure this is a direct comparison because Mastodon can't avoid doing this on-demand. If we're comparing, it would be good to list the tradeoffs of Mastodon's implementation (and how it scales to deep threads) more explicitly.
      - alethic 1y ago
        There is a detailed explanation available at the link I posted. Second header, "Approach".
        danabramov 1y ago
        What do you expect the performance characteristics to be compared to querying a database?
        alethic 1y ago
        I expect them to be unimportant. This has been merged upstream and running on the flagship Mastodon instance for a little while now. There is also a section related to performance available at the link I posted. Third header, "Likely Concerns", second subheader, "DoS/Amplification".
        danabramov 1y ago
        What do you mean by unimportant? I mean from the user's perspective: when I open a thread, I expect to instantly see the entire discussion happening across the entire network, with the paginated data coming back in a single roundtrip. Moreover, I expect every actor participating in the said discussion (wherever their data is stored) to see the same discussion as I do, with the same level of being "filled in", and in real time (each reply should immediately appear for each participant). It should be indistinguishable from UX of a centralized service where things happen instantly and are presented deterministically and universally (setting aside that centralized services abandoned these ideals in favor of personalization). With ATProto, this is clearly achieved (by reading already indexed information from the database). How can you achieve this expectation in an architecture where there's no single source of truth and you have to query different sources for different pieces on demand in a worker? (To clarify, I did read the linked PR. I'm asking you because it seems obviously unachievable to me, so I'm hoping you'll acknowledge this isn't a 1:1 comparison in terms of user experience.) To give a concrete example: is this really saying that replies will only be refreshed once in fifteen minutes[1]? The user expectation from centralized services is at most a few seconds. [1]: https://github.com/mastodon/mastodon/pull/32615/files#diff-6dc6e5fe84dacdb226ddb665f9b837adaafb3c3bc94f0079311ebf51049e88e4R97 https://github.com/mastodon/mastodon/pull/32615/files#diff-6...
    - INTPenis 1y ago
      Yes but it's off by default for a reason, it's not realistic to expect every node to fetch all messages. This is a catch 22 because the reason fedi is more decentralized is because of the low barrier of entry to run a node, but at the same time that low barrier means it takes less resources because it does not fetch every single message and piece of media.
      - alethic 1y ago
        It's currently on on the flagship instance and will be on by default in the upcoming 4.5 release.
        INTPenis 1y ago
        Creating more monolithic centralization. Us small instances can't afford it.
immibis 1y ago
I think that thinking of "the fediverse" as one thing at all is a mistake. It's Gilbert Ryle's classic category mistake: > A foreigner visiting Oxford or Cambridge for the first time is shown a number of colleges, libraries, playing fields, museums, scientific departments and administrative offices. He then asks “But where is the University?” ..... "The fediverse" is not a thing. There are many separate sites and the collection of all of them in total may be called "the fediverse", but that is not a thing by itself. They don't even all share a protocol in common. You cannot join the fediverse any more than you can join the game industry or the startup scene. You have to join a specific server, game company or startup (or more than one). And while from the outside you might have heard a lot of "the game industry is cut-throat" or "startups work together to innovate technology", once you are inside one of them, you'll find that it's a lot more fragmented than it at first appeared, and totally incohesive, and although from a bird's eye view it looked like you and your competitor down the hall were both making VR happen, they won't work together with you.

torben-friis 1y ago

What do you mean by consistent? (Never used the fediverse, so zero context here).

skybrian 1y ago
Not who you're replying to, but one issue is that different users will sometimes see a different subset of replies to a post, depending on what the server they're using has copied from other servers.
- alethic 1y ago
  This isn't correct. Mastodon merged fetch-all-replies in March. https://github.com/mastodon/mastodon/pull/32615 https://github.com/mastodon/mastodon/pull/32615 The only difference in visible replies is in the moderation choices of the server the post is viewed from.
  - skybrian 1y ago
    Good to know. Thanks for the update!
  - danabramov 1y ago
    How does it work? In ATProto, there is no need to do this on-demand because the data is already there in the AppView. When you want to serve a page of replies, you read them from the database and serve them. There is no distributed fetching involved, no need to hit someone else's servers, no need to coalesce them or worry about limiting fetches, etc. This is why it works fine for threads without thousands of replies and hundreds of nesting levels. It can also be paginated on the server. If you don't have this information on your server, how can you gracefully fetch thousands of replies from different servers and present a cohesive picture during a single request? I'm sure this PR does an attempt at that but I'm not sure this is a direct comparison because Mastodon can't avoid doing this on-demand. If we're comparing, it would be good to list the tradeoffs of Mastodon's implementation (and how it scales to deep threads) more explicitly.
    - alethic 1y ago
      There is a detailed explanation available at the link I posted. Second header, "Approach".
      - danabramov 1y ago
        What do you expect the performance characteristics to be compared to querying a database?
        alethic 1y ago
        I expect them to be unimportant. This has been merged upstream and running on the flagship Mastodon instance for a little while now. There is also a section related to performance available at the link I posted. Third header, "Likely Concerns", second subheader, "DoS/Amplification".
        danabramov 1y ago
        What do you mean by unimportant? I mean from the user's perspective: when I open a thread, I expect to instantly see the entire discussion happening across the entire network, with the paginated data coming back in a single roundtrip. Moreover, I expect every actor participating in the said discussion (wherever their data is stored) to see the same discussion as I do, with the same level of being "filled in", and in real time (each reply should immediately appear for each participant). It should be indistinguishable from UX of a centralized service where things happen instantly and are presented deterministically and universally (setting aside that centralized services abandoned these ideals in favor of personalization). With ATProto, this is clearly achieved (by reading already indexed information from the database). How can you achieve this expectation in an architecture where there's no single source of truth and you have to query different sources for different pieces on demand in a worker? (To clarify, I did read the linked PR. I'm asking you because it seems obviously unachievable to me, so I'm hoping you'll acknowledge this isn't a 1:1 comparison in terms of user experience.) To give a concrete example: is this really saying that replies will only be refreshed once in fifteen minutes[1]? The user expectation from centralized services is at most a few seconds. [1]: https://github.com/mastodon/mastodon/pull/32615/files#diff-6dc6e5fe84dacdb226ddb665f9b837adaafb3c3bc94f0079311ebf51049e88e4R97 https://github.com/mastodon/mastodon/pull/32615/files#diff-6...
  - INTPenis 1y ago
    Yes but it's off by default for a reason, it's not realistic to expect every node to fetch all messages. This is a catch 22 because the reason fedi is more decentralized is because of the low barrier of entry to run a node, but at the same time that low barrier means it takes less resources because it does not fetch every single message and piece of media.
    - alethic 1y ago
      It's currently on on the flagship instance and will be on by default in the upcoming 4.5 release.
      - INTPenis 1y ago
        Creating more monolithic centralization. Us small instances can't afford it.

immibis 1y ago

I think that thinking of "the fediverse" as one thing at all is a mistake. It's Gilbert Ryle's classic category mistake: > A foreigner visiting Oxford or Cambridge for the first time is shown a number of colleges, libraries, playing fields, museums, scientific departments and administrative offices. He then asks “But where is the University?” ..... "The fediverse" is not a thing. There are many separate sites and the collection of all of them in total may be called "the fediverse", but that is not a thing by itself. They don't even all share a protocol in common. You cannot join the fediverse any more than you can join the game industry or the startup scene. You have to join a specific server, game company or startup (or more than one). And while from the outside you might have heard a lot of "the game industry is cut-throat" or "startups work together to innovate technology", once you are inside one of them, you'll find that it's a lot more fragmented than it at first appeared, and totally incohesive, and although from a bird's eye view it looked like you and your competitor down the hall were both making VR happen, they won't work together with you.