3 ms·

That's a big post that doesn't really explain in what way can smaller players federate with ATProto or how the structure allows federation. Reading through it,

by izacus 1y ago

That's a big post that doesn't really explain in what way can smaller players federate with ATProto or how the structure allows federation. Reading through it, it just sounds like sharding/scaling for a centralized service that's meant to be owned and provided by a single entity.

danabramov 1y ago
>in what way can smaller players federate with ATProto or how the structure allows federation. Each of the pieces I've described (PDS, Relay, AppView) implement the protocol specified at https://atproto.com/ https://atproto.com/. Anything that acts as an ATProto PDS can be used as an ATProto PDS, anything that acts as an ATProto Relay can be used as an ATProto Relay, and so on. I'm not sure I understand the question so pardon the tautology. The structure allows federation by design — a Relay will index any PDS that asks to be indexed; an AppView can choose the Relay it wants to get the data from (or skip a Relay completely and index PDS's directly); anyone can make their own AppView for an existing or a new app. That's how there are multiple AppViews (both for Bluesky app and for other ATProto apps) ingesting data via multiple Relays from many PDS's. There aren't many independent operators of each piece (especially outside of PDS self-hosting) but nothing is privileging Bluesky's infra. Additionally, Bluesky's reference implementations of each piece are open source. So people run them the same way you would usually run software -- by putting it on a computer and exposing it to the internet. To run a custom PDS, you can either use the Docker container provided by Bluesky (https://github.com/bluesky-social/pds https://github.com/bluesky-social/pds) or implement your own (e.g. https://github.com/blacksky-algorithms/rsky https://github.com/blacksky-algorithms/rsky). Ditto for other pieces. >Reading through it, it just sounds like sharding/scaling for a centralized service that's meant to be owned and provided by a single entity. You're right in that the goal is to make it on par with centralized services in terms of UX and performance/scaling. However, it is decentralized. The picture at the end of this article might help: https://atproto.com/articles/atproto-for-distsys-engineers https://atproto.com/articles/atproto-for-distsys-engineers
- HexDecOctBin 1y ago
  I think what they are asking is: if I run a my own BlueSky AppView, how do I integrate with the bluesky.app so that users signed in on my AppView can interact with users on the the main AppView and vice versa? This is how most of us think about federalisation.
  - danabramov 1y ago
    That's already the default behavior. You don't need to do anything special for that to work, it's what ATProto is designed for. Everything is always operating in a shared space by design. When people "post", their posts go to their PDS's, which means that every AppView ingests data generated by every other AppView by default. There is no way to tell who's using which AppView — in fact, you can log into any AppView and your profile will be there with all your posts.
  - jauntywundrkind 1y ago
    AppViews do things like track likes, or replies. The AppView doesn't do that only for Bluesky data. It does it for any Personal Data Stores (user accounts with all their user data) that it knows about. When you "interact" with users elsewhere, all you do is generate new records on your own PDS. You generate a "like" entry, or a reply, on your own PDS. It's your pds, all your stuff goes there. The AppView sees that and indexes it, attaches that like or that reply in the AppView to the post you're reacting to.
  - OneDeuxTriSeiGo 1y ago
    An appview is really just a service for hydrating content in the skeleton of posts that feeds return back to give to the user. The only things they do other than feed hydration are track notifications, (optionally) provide a search engine, (optionally) provide a CDN, and (temporarily until E2EE rolls out) handle DMs. So you can actually do things like the Red Dwarf [1] project which is a bluesky client without an appview. It's slower, you visibly notice request loading/pop-in, there's no notifications, and no search but it works with any other bluesky appview (since appviews are basically a lens into atproto rather than an independent service). -------- If you wanted to run your own infrastructure, instead you'd probably want to run your own PDS. Running an appview has its benefits of course but the main way you "self host" is to run a PDS. That's fairly trivial and people have run them on all kinds of constrained hardware (including a literal jailbroken microwave if I remember correctly). 1. https://tangled.sh/@whey.party/red-dwarf https://tangled.sh/@whey.party/red-dwarf
AgentME 1y ago
Other people can run AppViews too (that operate over the same or different data). There are just less people doing that than hosting Mastodon instances partly because it's much more expensive to, because some of the benefits of hosting a Mastodon instance can be obtained much cheaper through running a PDS server, and because AppViews doesn't serve the same exact social role that Mastodon instances do. (Mastodon has every instance be a semi-isolated community, so Mastodon instances are often made for the social purpose of running a semi-isolated community. Bluesky users expect a global timeline that's not partitioned by server instances, so it doesn't get many people running AppViews specifically for fostering semi-isolated communities. People on Bluesky who want to foster semi-isolated communities tend to use features like custom timelines to do so instead.)
EnglishMobster 1y ago
I think it can be explained a little better. When you write a post, you save it to your PDS. Think of it like writing a blog. You're done, you hit submit, it shows up on a server somewhere. You can run your own server with your own data, or use someone else's. That's exactly how a PDS works; it is a storage server for your data. The AppView is a way to index all the PDSes registered across the whole network. If your server is crawlable by the AppView, all your data shows up in the app. This is like if your blog is crawlable by Google, you show up in search results. When you like a post, you commit a "like" record to your personal server. When the AppView displays likes, it looks at every indexed PDS and shows every like it can find for that post (simplifying a little for clarity). Each one of those likes might live on different servers, some of which is self-hosted. Because you can run your own PDS, you can commit any data you want to it. You can even commit things that services may find distasteful. However, the AppView may refuse to serve this content to users; this is how content can be removed from the network and how users can be banned. The federation equivalent would be defederation, except it happens to singular accounts rather than entire instances. If you disagree with the moderation policies run by Bluesky the company, that's when you can look into running an alternative AppView. This is similar to disagreeing with the admin of a particular Mastodon instance and moving to a different instance. Of course, as mentioned running an AppView is much more expensive, but that hasn't stopped folks from trying (I believe Blacksky is trying to run their own AppView that is fully independent of Bluesky). To use an alternate AppView, you'd simply go to a different website. This website will index PDSes the same way that Bluesky does, but it may index them in a different way and include/exclude different content. The data is still there (nobody can reach into your PDS on your server and delete your data), but the AppView admins choose which content they wish to serve to people using their community, just as Mastodon admins choose who to federate with. In this sense, it is indeed truly federated. The primitives are simply different; it's more granular than Mastodon. You can write your own content to your own server and let it get indexed to any number of AppViews; you completely control your personal data and nobody can reach in and delete that data randomly as they don't own it - you do (at the cost of ~$1/month or a Raspberry Pi). When you use the Bluesky service, you are seeing their view of the network and what they choose to index. You may disagree with this view, just as you may disagree with the admins of Mastodon.social etc. In that case, you can choose to use another AppView (such as deer.social or Blacksky) that adopts different policies. Since account information isn't stored on the AppView and it simply handles indexing and moderation, moving between AppViews is painless and no data needs to be transferred from one server to another - you simply use a different bookmark. It could be that you disagree with all the current AppView admins. You can host your own, it's just expensive ($300/month, as mentioned). You can also tailor your AppView to index less content, which will of course limit the amount of data you consume and give you a partial view of the network, effectively defederating you from anything you do not wish to index. But there's nothing stopping you from doing so!

by izacus 1y ago

danabramov 1y ago

>in what way can smaller players federate with ATProto or how the structure allows federation. Each of the pieces I've described (PDS, Relay, AppView) implement the protocol specified at https://atproto.com/ https://atproto.com/. Anything that acts as an ATProto PDS can be used as an ATProto PDS, anything that acts as an ATProto Relay can be used as an ATProto Relay, and so on. I'm not sure I understand the question so pardon the tautology. The structure allows federation by design — a Relay will index any PDS that asks to be indexed; an AppView can choose the Relay it wants to get the data from (or skip a Relay completely and index PDS's directly); anyone can make their own AppView for an existing or a new app. That's how there are multiple AppViews (both for Bluesky app and for other ATProto apps) ingesting data via multiple Relays from many PDS's. There aren't many independent operators of each piece (especially outside of PDS self-hosting) but nothing is privileging Bluesky's infra. Additionally, Bluesky's reference implementations of each piece are open source. So people run them the same way you would usually run software -- by putting it on a computer and exposing it to the internet. To run a custom PDS, you can either use the Docker container provided by Bluesky (https://github.com/bluesky-social/pds https://github.com/bluesky-social/pds) or implement your own (e.g. https://github.com/blacksky-algorithms/rsky https://github.com/blacksky-algorithms/rsky). Ditto for other pieces. >Reading through it, it just sounds like sharding/scaling for a centralized service that's meant to be owned and provided by a single entity. You're right in that the goal is to make it on par with centralized services in terms of UX and performance/scaling. However, it is decentralized. The picture at the end of this article might help: https://atproto.com/articles/atproto-for-distsys-engineers https://atproto.com/articles/atproto-for-distsys-engineers

HexDecOctBin 1y ago
I think what they are asking is: if I run a my own BlueSky AppView, how do I integrate with the bluesky.app so that users signed in on my AppView can interact with users on the the main AppView and vice versa? This is how most of us think about federalisation.
- danabramov 1y ago
  That's already the default behavior. You don't need to do anything special for that to work, it's what ATProto is designed for. Everything is always operating in a shared space by design. When people "post", their posts go to their PDS's, which means that every AppView ingests data generated by every other AppView by default. There is no way to tell who's using which AppView — in fact, you can log into any AppView and your profile will be there with all your posts.
- jauntywundrkind 1y ago
  AppViews do things like track likes, or replies. The AppView doesn't do that only for Bluesky data. It does it for any Personal Data Stores (user accounts with all their user data) that it knows about. When you "interact" with users elsewhere, all you do is generate new records on your own PDS. You generate a "like" entry, or a reply, on your own PDS. It's your pds, all your stuff goes there. The AppView sees that and indexes it, attaches that like or that reply in the AppView to the post you're reacting to.
- OneDeuxTriSeiGo 1y ago
  An appview is really just a service for hydrating content in the skeleton of posts that feeds return back to give to the user. The only things they do other than feed hydration are track notifications, (optionally) provide a search engine, (optionally) provide a CDN, and (temporarily until E2EE rolls out) handle DMs. So you can actually do things like the Red Dwarf [1] project which is a bluesky client without an appview. It's slower, you visibly notice request loading/pop-in, there's no notifications, and no search but it works with any other bluesky appview (since appviews are basically a lens into atproto rather than an independent service). -------- If you wanted to run your own infrastructure, instead you'd probably want to run your own PDS. Running an appview has its benefits of course but the main way you "self host" is to run a PDS. That's fairly trivial and people have run them on all kinds of constrained hardware (including a literal jailbroken microwave if I remember correctly). 1. https://tangled.sh/@whey.party/red-dwarf https://tangled.sh/@whey.party/red-dwarf

AgentME 1y ago

Other people can run AppViews too (that operate over the same or different data). There are just less people doing that than hosting Mastodon instances partly because it's much more expensive to, because some of the benefits of hosting a Mastodon instance can be obtained much cheaper through running a PDS server, and because AppViews doesn't serve the same exact social role that Mastodon instances do. (Mastodon has every instance be a semi-isolated community, so Mastodon instances are often made for the social purpose of running a semi-isolated community. Bluesky users expect a global timeline that's not partitioned by server instances, so it doesn't get many people running AppViews specifically for fostering semi-isolated communities. People on Bluesky who want to foster semi-isolated communities tend to use features like custom timelines to do so instead.)

EnglishMobster 1y ago

I think it can be explained a little better. When you write a post, you save it to your PDS. Think of it like writing a blog. You're done, you hit submit, it shows up on a server somewhere. You can run your own server with your own data, or use someone else's. That's exactly how a PDS works; it is a storage server for your data. The AppView is a way to index all the PDSes registered across the whole network. If your server is crawlable by the AppView, all your data shows up in the app. This is like if your blog is crawlable by Google, you show up in search results. When you like a post, you commit a "like" record to your personal server. When the AppView displays likes, it looks at every indexed PDS and shows every like it can find for that post (simplifying a little for clarity). Each one of those likes might live on different servers, some of which is self-hosted. Because you can run your own PDS, you can commit any data you want to it. You can even commit things that services may find distasteful. However, the AppView may refuse to serve this content to users; this is how content can be removed from the network and how users can be banned. The federation equivalent would be defederation, except it happens to singular accounts rather than entire instances. If you disagree with the moderation policies run by Bluesky the company, that's when you can look into running an alternative AppView. This is similar to disagreeing with the admin of a particular Mastodon instance and moving to a different instance. Of course, as mentioned running an AppView is much more expensive, but that hasn't stopped folks from trying (I believe Blacksky is trying to run their own AppView that is fully independent of Bluesky). To use an alternate AppView, you'd simply go to a different website. This website will index PDSes the same way that Bluesky does, but it may index them in a different way and include/exclude different content. The data is still there (nobody can reach into your PDS on your server and delete your data), but the AppView admins choose which content they wish to serve to people using their community, just as Mastodon admins choose who to federate with. In this sense, it is indeed truly federated. The primitives are simply different; it's more granular than Mastodon. You can write your own content to your own server and let it get indexed to any number of AppViews; you completely control your personal data and nobody can reach in and delete that data randomly as they don't own it - you do (at the cost of ~$1/month or a Raspberry Pi). When you use the Bluesky service, you are seeing their view of the network and what they choose to index. You may disagree with this view, just as you may disagree with the admins of Mastodon.social etc. In that case, you can choose to use another AppView (such as deer.social or Blacksky) that adopts different policies. Since account information isn't stored on the AppView and it simply handles indexing and moderation, moving between AppViews is painless and no data needs to be transferred from one server to another - you simply use a different bookmark. It could be that you disagree with all the current AppView admins. You can host your own, it's just expensive ($300/month, as mentioned). You can also tailor your AppView to index less content, which will of course limit the amount of data you consume and give you a partial view of the network, effectively defederating you from anything you do not wish to index. But there's nothing stopping you from doing so!