5 ms·

didn't say it was cheap!

by __justplaying 2y ago

bombcar 2y ago
Ten terabytes of spinning rust is only $100-$300 or so, that's not bad at all.
- jonstaab 2y ago
  My point is not the current size, it's the eventual size if bluesky succeeds. Facebook ingests 100TB/day. Self-hosting a bluesky relay isn't (won't be) a thing.
  - galactus 2y ago
    It could be a thing. Not for individual tinkerers but for companies. The fact that today, with already 14 million users, is still possible for an individual to host it is amazing.
nightpool 2y ago
But why is it required? Do you really need a copy of everyone's data locally? If the only way to self-host bluesky is to have an entire copy of the entire database, that seems like it's really bad from a scaling perspective.
- half-kh-hacker 2y ago
  What else would "self-hosting all of Bluesky" mean other than a copy of the entire site? If you just want to participate in the network host a PDS, which only stores your own posts.
  - nightpool 2y ago
    Surely there's some middle ground between only hosting your own data and being reliant on another site to keep track of your following / followers and hosting a duplicate copy of the entire network?
    - steveklabnik 2y ago
      For sure. If you just want to host your own data, you can do that. A PDS for you and maybe some friends is very small and cheap to host.
      - nightpool 2y ago
        My understanding though is that having a PDS on its own is useless without an AppView to collect the data from the relay? Or am I misunderstanding the architecture here? https://docs.bsky.app/docs/advanced-guides/federation-architecture https://docs.bsky.app/docs/advanced-guides/federation-archit...
        steveklabnik 2y ago
        I'm talking about the case where you wanted to run your own PDS and use all of the other infrastructure being run by Bluesky. If you fully want your own copy of everything, then you'd want to run a copy of everything. But you don't have to. It really depends on what your goals are. That's why the post is about the maximal scenario. "Just your own PDS" is the minimalist scenario. But I think it's the one that makes sense for 95% of users who want to self-host.
        nightpool 2y ago
        Right, and I'm saying "surely there must be a middle ground between "using all of Bluesky's infrastructure" and "having a 4.5tb copy of every post ever made on the network""
        lisowski 2y ago
        What exactly would that be? I feel like the middle ground your talking about could be just a feed? A feed is: a server that consumes the firehose and decided on whether to store posts, when loaded in the app it returns some post to create a feed So essentially you only store references to part of the network rather than storing the whole thing
- galactus 2y ago
  Uh, it is not required. You can run only a PDS if you want to self host your data and everything will work. But it is indeed very cool that you can actually host a relay if you want (for fun, learning, or whatever reason)
- jazzyjackson 2y ago
  "self host an entire copy of all user data" is a pretty cool capability to have, kind of proof that the infrastructure is really open and forkable. you seem to have misunderstood OPs goals. Serving your own data from a personal data server is a much less arduous affair.

didn't say it was cheap!

bombcar 2y ago

Ten terabytes of spinning rust is only $100-$300 or so, that's not bad at all.

jonstaab 2y ago
My point is not the current size, it's the eventual size if bluesky succeeds. Facebook ingests 100TB/day. Self-hosting a bluesky relay isn't (won't be) a thing.
- galactus 2y ago
  It could be a thing. Not for individual tinkerers but for companies. The fact that today, with already 14 million users, is still possible for an individual to host it is amazing.

nightpool 2y ago

But why is it required? Do you really need a copy of everyone's data locally? If the only way to self-host bluesky is to have an entire copy of the entire database, that seems like it's really bad from a scaling perspective.

half-kh-hacker 2y ago
What else would "self-hosting all of Bluesky" mean other than a copy of the entire site? If you just want to participate in the network host a PDS, which only stores your own posts.
- nightpool 2y ago
  Surely there's some middle ground between only hosting your own data and being reliant on another site to keep track of your following / followers and hosting a duplicate copy of the entire network?
  - steveklabnik 2y ago
    For sure. If you just want to host your own data, you can do that. A PDS for you and maybe some friends is very small and cheap to host.
    - nightpool 2y ago
      My understanding though is that having a PDS on its own is useless without an AppView to collect the data from the relay? Or am I misunderstanding the architecture here? https://docs.bsky.app/docs/advanced-guides/federation-architecture https://docs.bsky.app/docs/advanced-guides/federation-archit...
      - steveklabnik 2y ago
        I'm talking about the case where you wanted to run your own PDS and use all of the other infrastructure being run by Bluesky. If you fully want your own copy of everything, then you'd want to run a copy of everything. But you don't have to. It really depends on what your goals are. That's why the post is about the maximal scenario. "Just your own PDS" is the minimalist scenario. But I think it's the one that makes sense for 95% of users who want to self-host.
        nightpool 2y ago
        Right, and I'm saying "surely there must be a middle ground between "using all of Bluesky's infrastructure" and "having a 4.5tb copy of every post ever made on the network""
        lisowski 2y ago
        What exactly would that be? I feel like the middle ground your talking about could be just a feed? A feed is: a server that consumes the firehose and decided on whether to store posts, when loaded in the app it returns some post to create a feed So essentially you only store references to part of the network rather than storing the whole thing
galactus 2y ago
Uh, it is not required. You can run only a PDS if you want to self host your data and everything will work. But it is indeed very cool that you can actually host a relay if you want (for fun, learning, or whatever reason)
jazzyjackson 2y ago
"self host an entire copy of all user data" is a pretty cool capability to have, kind of proof that the infrastructure is really open and forkable. you seem to have misunderstood OPs goals. Serving your own data from a personal data server is a much less arduous affair.