4 ms·

The atproto team came from the p2p space. We had a lot of experience running client-side computation. There are challenges you can try to solve in that design -

by pfraze 2y ago

The atproto team came from the p2p space. We had a lot of experience running client-side computation. There are challenges you can try to solve in that design -- key sync, key recovery, reliable hosting, reliable discovery, etc -- but what you can't get around is the need to run queries against aggregations of data. Even scales that we consider "mid" start to strain on the client-driven model. Federated queries might be able to solve it, but it's extremely challenging to get reliable performance out of that. The design we landed on was replaceable big nodes. The common comparable people raise is email/gmail, but we used the DID system and account portability to try to get a better outcome on provider migration. It's hopefully more like web/google -- which still has the centralizing pressures of scale benefits, but hopefully injects enough fluidity in the system to move the situation forward. Sometimes, you pick the design that moves the ball down the field, not the one that guarantees a touchdown. If somebody wanted to improve on what we've done, I'd tell them to focus on the federated queries problem.

jchw 2y ago
In theory AT proto doesn't seem like a bad design. I've read a fair bit of the docs, although mostly skimming. (I've been meaning to read the paper on Merkle Search Trees so I can figure out what exactly is going on with PDSes.) On the other hand, in practice it seems like the AT proto infrastructure is still very centralized for now. DIDs are excellent in theory, but everyone is using PLC DIDs, which depends on the centralized plc.directory. You can run your own PDSes, but there's only one relay for now (that I am aware of.) I also don't think there is more than one instance of the Bluesky AppView, and the official instance is locked into the Bluesky Moderation Service, which seems to limit the usefulness of some of the censorship resistance of the protocol. I'm not sure how much of that is social problems rather than technical, but I worry that since Bluesky and AT proto are gaining massive popularity with this status quo (millions of users!) it'll have a problem akin to Matrix.org, where in practice almost everyone is using the same infrastructure anyways. It's still relatively early days, but millions of people is definitely enough to where you start to hit problems with having everyone under one roof. I really hope we get to see how things play out when more of the network is operated independently.
- Arathorn 2y ago
  fwiw roughly 50% of Matrix is on the matrix.org instance currently. we consider this a bug, but also prioritise ease of onboarding over decentralisation purity ideology.
  - jchw 2y ago
    I hate to be a downer but there's a lot of things Matrix could prioritize over decentralization. That said, the decentralization works pretty badly. Large federated joins are somewhere between comically slow and horrifically slow. Status does not seem to work well across federation either. I'm also a bit miffed that Dendrite was positioned as a "next generation" Matrix server but now it feels nearly orphaned with missing support for newer features, issues with various appservice bridges, few updates at a very slow pace, and no migration path out in sight. I know it came with a fair number of disclaimers, but that still bums me out as it seemed like it would be okay for a small non-critical homeserver, and now it seems likely I'll have to engineer my own path out when clients finally stop working with Dendrite. (It already happened once...) You have no idea how bad I want to love Matrix, but frankly if it was due to a focus on usability that decentralization "purity" has suffered, it simply does not show in the resulting usability improvements over years of time. Sorry to be harsh.
    - Arathorn 2y ago
      if you feel miffed, imagine how the Dendrite team feels, given the lack of funding which means it is on best-effort dev currently. if you’re interested in progress on Matrix, https://matrix.org/blog/2024/10/29/matrix-2.0-is-here/ https://matrix.org/blog/2024/10/29/matrix-2.0-is-here/ is where it’s at.
      - jchw 2y ago
        I did see the Matrix 2.0 announcement, though for obvious reasons I can't actually use any of the features listed in it. Obviously, improvements to the basic chat functions of Matrix would be great. For now though, I am stuck with the reality that joining a large federated channel sometimes takes more than 6 hours. I wish I were exaggerating. edit: I guess though that faster room joins weren't a part of Matrix 2.0. Actually, I don't really have a huge problem with the sync taking too long personally. So maybe Matrix 2.0 wouldn't bring that big of an improvement for me anyway.

by pfraze 2y ago

jchw 2y ago

In theory AT proto doesn't seem like a bad design. I've read a fair bit of the docs, although mostly skimming. (I've been meaning to read the paper on Merkle Search Trees so I can figure out what exactly is going on with PDSes.) On the other hand, in practice it seems like the AT proto infrastructure is still very centralized for now. DIDs are excellent in theory, but everyone is using PLC DIDs, which depends on the centralized plc.directory. You can run your own PDSes, but there's only one relay for now (that I am aware of.) I also don't think there is more than one instance of the Bluesky AppView, and the official instance is locked into the Bluesky Moderation Service, which seems to limit the usefulness of some of the censorship resistance of the protocol. I'm not sure how much of that is social problems rather than technical, but I worry that since Bluesky and AT proto are gaining massive popularity with this status quo (millions of users!) it'll have a problem akin to Matrix.org, where in practice almost everyone is using the same infrastructure anyways. It's still relatively early days, but millions of people is definitely enough to where you start to hit problems with having everyone under one roof. I really hope we get to see how things play out when more of the network is operated independently.

Arathorn 2y ago
fwiw roughly 50% of Matrix is on the matrix.org instance currently. we consider this a bug, but also prioritise ease of onboarding over decentralisation purity ideology.
- jchw 2y ago
  I hate to be a downer but there's a lot of things Matrix could prioritize over decentralization. That said, the decentralization works pretty badly. Large federated joins are somewhere between comically slow and horrifically slow. Status does not seem to work well across federation either. I'm also a bit miffed that Dendrite was positioned as a "next generation" Matrix server but now it feels nearly orphaned with missing support for newer features, issues with various appservice bridges, few updates at a very slow pace, and no migration path out in sight. I know it came with a fair number of disclaimers, but that still bums me out as it seemed like it would be okay for a small non-critical homeserver, and now it seems likely I'll have to engineer my own path out when clients finally stop working with Dendrite. (It already happened once...) You have no idea how bad I want to love Matrix, but frankly if it was due to a focus on usability that decentralization "purity" has suffered, it simply does not show in the resulting usability improvements over years of time. Sorry to be harsh.
  - Arathorn 2y ago
    if you feel miffed, imagine how the Dendrite team feels, given the lack of funding which means it is on best-effort dev currently. if you’re interested in progress on Matrix, https://matrix.org/blog/2024/10/29/matrix-2.0-is-here/ https://matrix.org/blog/2024/10/29/matrix-2.0-is-here/ is where it’s at.
    - jchw 2y ago
      I did see the Matrix 2.0 announcement, though for obvious reasons I can't actually use any of the features listed in it. Obviously, improvements to the basic chat functions of Matrix would be great. For now though, I am stuck with the reality that joining a large federated channel sometimes takes more than 6 hours. I wish I were exaggerating. edit: I guess though that faster room joins weren't a part of Matrix 2.0. Actually, I don't really have a huge problem with the sync taking too long personally. So maybe Matrix 2.0 wouldn't bring that big of an improvement for me anyway.