5 ms·

I run a single user ActivityPub instance with a minimal following and small number of people across multiple instances that I follow. From a user perspective Ac

by zaphar 3y ago

I run a single user ActivityPub instance with a minimal following and small number of people across multiple instances that I follow. From a user perspective ActivityPub is fine I have no complaints. However from an Ops perspective ActivityPub is incredibly chatty. If this had to scale to a larger instance the costs would spiral fast. Operationally and cost efficiency wise ATProto is a better looking protocol already. From a single individual user this won't necessarily be obvious right off the bat. But it will tend to manifest in either overworked operations people or slow janky instance performance. While it's certainly a reasonable question whether the world needs another federated social protocol or not ATProto definitely solves real problems with the ActivityPub protocol.

lucideer 3y ago
> ActivityPub is incredibly chatty. > ATProto is a better looking protocol already. Are there benchmarks for this? What's the level of difference here? Request frequency seems closely linked to activity and XRPC bodies are JSON just as ActivityPub so message size should be within order of magnitude at least. Are there architectural differences that reduce request frequency significantly? > If this had to scale to a larger instance the costs would spiral fast. Are we talking bandwidth costs or processing. I know the popular ActivityPub implementation is widely considered to be pretty inefficient processing-wise for reasons unrelated to the protocol itself: is that a factor here?
- mkr-hn 3y ago
  Sidekiq seems to be the culprit from what I've seen and read from people who've run into this issue. It gets overloaded fast if you don't have enough processing power in front of the queue. Lighter implementations apparently do something different, or are more efficient in handling their queues without whatever overhead Sidekiq adds.
- zaphar 3y ago
  One example of the chattiness is a flow where more than one person is following the same individual on another server. That person will have to push new messages to every single one of the people following them. This means that if 10 users are following me from the same server I will not have 1 push for that instance, I'll gave 10 pushes for the same single unchanged message. This is built into the protocol. That's a lot of throughput for something that could be much much less chatty. Now on my single instance it's not too bad because 1. I follow maybe 40 people and 2. I have like max 10 followers. For an instance with people with high follower counts across multiple other instances it could get to be a problem fast. Edit: my previous description used fetch when it should have used push.
  - Spivak 3y ago
    Isn't that "more than once instance" rather than "more than one user"? I think the weirdness is with Bluesky all that cost is still there but it's now handled by a small group of massive megacorps which is a real tangible benefit to self-hosters but you could have that on top of AP by running your service off what would essentially be a massive global cache of AP content which is what the indexer is.
    - zaphar 3y ago
      Yes, I edited my comment too be more clear and. I shouldn't have referred to it as fetch because it's actually push.
      - zaphar 3y ago
        I should note that as a protocol I suspect that ATProto is less chatty which does translate to reduced costs. It adds features on top that some may or may not want which increase costs in other ways but only for the people who want to utilize those features. It's not exactly an Apples to Apples comparison.
  - lucideer 3y ago
    > This is built into the protocol. I haven't implemented AP from scratch so I may be missing technical details precluding this, but it sounds like it may at least partially be covered by https://www.w3.org/TR/activitypub/#shared-inbox-delivery https://www.w3.org/TR/activitypub/#shared-inbox-delivery ... in which case it may be an implementation issue? Mind you, there is liberal use of "MAY" there which I find is always a problem with specs: that would likely lead to mandatory chattiness of outgoing requests if you're federated with a lot of instances without shared inbox support, but should at least solve for incoming.
    - zaphar 3y ago
      In theory it would help. In practice since a server can't rely on this it probably devolves to just ignoring that feature.
DannyBee 3y ago
Being technically better is usually not a good enough reason to be incompatible. I'm not sure why people don't get this, but it is almost always true. Starting from scratch, just because you can theoretically design a better system, is one of the worst thing to do to users. Theoretically better also rarely wins in the marketplace anyway. If you want a slightly lighter position: Software needs to be built to be migrated from and to, as a base level requirement. Those communities that do this tend end up with happy users who don't spend lots of toil trying to keep up. This is true regardless of whether the new thing is compatible with the old - if it doesn't take any energy or time, and just works, people care less what you change. Yes, it is hard. yes, there are choices you make sometimes that burn you down the road. That is 100% guaranteed to happen. So make software that enable change to occur. The overall the amount of developer toil and waste created en masse by people who think they are making something "better", usually with no before/after data (or at best, small useless metrics sampled from a very small group), almost always vastly dwarfs all improvement that occurs as a result. If you want to spend time helping developers/users, then understand where they spend their time, not where you spend your time.
- zaphar 3y ago
  I wholeheartedly agree. From a purely technical analysis ATProto looks better as a protocol to me. But I don't use Bluesky I use ActivityPub because the people I want to be connected to are there and not on Bluesky. I do think you could probably make improvements to ActivityPub that reduce operational costs. It's not something I feel the need to tackle right this moment because my usage doesn't incure those costs really.
- comte7092 3y ago
  Something that I think your analysis is missing is that with a decentralized product, ops is also user experience. The whole point is that it needs to be reasonably easy for people to run and scale their own servers. If people are constantly being burned out, quit, or run out of money, then it has an impact on regular users.
  - DannyBee 3y ago

by zaphar 3y ago

lucideer 3y ago

> ActivityPub is incredibly chatty. > ATProto is a better looking protocol already. Are there benchmarks for this? What's the level of difference here? Request frequency seems closely linked to activity and XRPC bodies are JSON just as ActivityPub so message size should be within order of magnitude at least. Are there architectural differences that reduce request frequency significantly? > If this had to scale to a larger instance the costs would spiral fast. Are we talking bandwidth costs or processing. I know the popular ActivityPub implementation is widely considered to be pretty inefficient processing-wise for reasons unrelated to the protocol itself: is that a factor here?

mkr-hn 3y ago
Sidekiq seems to be the culprit from what I've seen and read from people who've run into this issue. It gets overloaded fast if you don't have enough processing power in front of the queue. Lighter implementations apparently do something different, or are more efficient in handling their queues without whatever overhead Sidekiq adds.
zaphar 3y ago
One example of the chattiness is a flow where more than one person is following the same individual on another server. That person will have to push new messages to every single one of the people following them. This means that if 10 users are following me from the same server I will not have 1 push for that instance, I'll gave 10 pushes for the same single unchanged message. This is built into the protocol. That's a lot of throughput for something that could be much much less chatty. Now on my single instance it's not too bad because 1. I follow maybe 40 people and 2. I have like max 10 followers. For an instance with people with high follower counts across multiple other instances it could get to be a problem fast. Edit: my previous description used fetch when it should have used push.
- Spivak 3y ago
  Isn't that "more than once instance" rather than "more than one user"? I think the weirdness is with Bluesky all that cost is still there but it's now handled by a small group of massive megacorps which is a real tangible benefit to self-hosters but you could have that on top of AP by running your service off what would essentially be a massive global cache of AP content which is what the indexer is.
  - zaphar 3y ago
    Yes, I edited my comment too be more clear and. I shouldn't have referred to it as fetch because it's actually push.
    - zaphar 3y ago
      I should note that as a protocol I suspect that ATProto is less chatty which does translate to reduced costs. It adds features on top that some may or may not want which increase costs in other ways but only for the people who want to utilize those features. It's not exactly an Apples to Apples comparison.
- lucideer 3y ago
  > This is built into the protocol. I haven't implemented AP from scratch so I may be missing technical details precluding this, but it sounds like it may at least partially be covered by https://www.w3.org/TR/activitypub/#shared-inbox-delivery https://www.w3.org/TR/activitypub/#shared-inbox-delivery ... in which case it may be an implementation issue? Mind you, there is liberal use of "MAY" there which I find is always a problem with specs: that would likely lead to mandatory chattiness of outgoing requests if you're federated with a lot of instances without shared inbox support, but should at least solve for incoming.
  - zaphar 3y ago
    In theory it would help. In practice since a server can't rely on this it probably devolves to just ignoring that feature.

DannyBee 3y ago

Being technically better is usually not a good enough reason to be incompatible. I'm not sure why people don't get this, but it is almost always true. Starting from scratch, just because you can theoretically design a better system, is one of the worst thing to do to users. Theoretically better also rarely wins in the marketplace anyway. If you want a slightly lighter position: Software needs to be built to be migrated from and to, as a base level requirement. Those communities that do this tend end up with happy users who don't spend lots of toil trying to keep up. This is true regardless of whether the new thing is compatible with the old - if it doesn't take any energy or time, and just works, people care less what you change. Yes, it is hard. yes, there are choices you make sometimes that burn you down the road. That is 100% guaranteed to happen. So make software that enable change to occur. The overall the amount of developer toil and waste created en masse by people who think they are making something "better", usually with no before/after data (or at best, small useless metrics sampled from a very small group), almost always vastly dwarfs all improvement that occurs as a result. If you want to spend time helping developers/users, then understand where they spend their time, not where you spend your time.

zaphar 3y ago
I wholeheartedly agree. From a purely technical analysis ATProto looks better as a protocol to me. But I don't use Bluesky I use ActivityPub because the people I want to be connected to are there and not on Bluesky. I do think you could probably make improvements to ActivityPub that reduce operational costs. It's not something I feel the need to tackle right this moment because my usage doesn't incure those costs really.
comte7092 3y ago
Something that I think your analysis is missing is that with a decentralized product, ops is also user experience. The whole point is that it needs to be reasonably easy for people to run and scale their own servers. If people are constantly being burned out, quit, or run out of money, then it has an impact on regular users.
- DannyBee 3y ago