15 ms·
Why isn't Bluesky a peer-to-peer network?
- deleted 3y ago[deleted]
- skybrian 3y agoThe author compares this model to the web, but it also sounds something like GitHub, but with more cryptography. Users publish their own repositories, but you still need big, expensive services to index it all and help people find each other. Currently the BlueSky search engine is pretty bad; it only goes back a few hours. If you don't post for a day, none of your posts will be found. Searching on obscure terms doesn't work very well.
- pfraze 3y agoDon't look at our search service as deeply correlated to the indexing model. That's not the point of the comparison, and our search service is quite subpar. We've only had time to put an ElasticSearch instance against the data, do some basic tuning, and then keep it online while we focus elsewhere.
- crotchfire 3y ago> you still need big, expensive services to help people find each other No, you really don't.
- jazzyjackson 3y agoThe BlueSky architecture takes it for granted that you do. That's why they split up "personal data servers" and relays (fka "big graph service") If you have 10 million people all spread out across small servers, how do you suggest I find everyone posting a particular hashtag?
- _heimdall 3y agoLinks worked well enough for the web, Google just came along and painted a picture of being able to find whatever you want whenever you want it. Social graphs can offer a similar discoverability without search. You see what those you follow post, comment on, like, share, etc. Following tags just like you follow a person also works well. Before assuming we must have to have massive indexing and searching services, ask yourself whether you really need to be able to discover anything that anyone around the world posts, regardless of whether you have any sort of connection to that person, their context, their society, or their culture.
- realfeel78 3y agoYou or I might not on a given day depending on our particular limited purposes, but the greater world around us absolutely does.
- _heimdall 3y agoI'm actually curious, what's a use case that really makes the broader system better by being able to discover content from anyone, anywhere? The best I can come up with is a historical record. Beyond that I'm at a loss for a use case that is noticeably better with search compared to social graph-based discovery.
- edgyquant 3y agoHow do you propose to see all of this without downloading that info?
- _heimdall 3y agoI'm not quite sure what you mean, sorry. My point was that something like BlueSky can make content more discoverable by letting you build your own social graph and view content from those you're connected to. Search is most useful for discovering content that you otherwise have no connection to at all. There is also the historical aspect and there really isn't a way around that, but in the original context of finding people actively talking about a specific topic it isn't a necessarily must-have feature.
- deleted 3y ago[deleted]
- evbogue 3y agoI appreciate the history because I've been through all of it. But, I don't think you guys are right. You can have a distributed system that also has authenticated posts. It's been done before, and it will be done again. And you can delete things. You just gotta trust nodes to also delete them and they probably won't. And I'm totally willing to help you guys build it. I know, you don't want me to work for you guys because I work at California-based company at the mall. But I assure you, I can get you guys over this little obstacle you've discovered. Call me! I'll answer 773-510-8601. Or the usual channels.
- josephg 3y agoSounds like they’ve found a solution to this problem that they’re happy with. Why not do it the way they’re doing it?
- evbogue 3y agoGood point!
- jazzyjackson 3y agohow are you doing peer discovery on bogbook? Do I have to find a contact's public key out-of-band before syncing their messages?
- jokethrowaway 3y agoThe distributed web works (worked?) because search engines had economic incentives to help users discover content, thanks to ads and they invested a lot of resources, which culminated in Google. I still have to see a distributed social network which works well enough for end users. It's not simple if you're centralising everything, no wonder distributed alternatives are hard to come by.
- deleted 3y ago[deleted]
- djbusby 3y agoWhy isn't ActivityPub more peer-to-peer? Like, it's all in a few dominant nodes (eg: mastodon). Shouldn't it be 1000s of little nodes (community) or individual publishers?
- MiguelX413 3y agoThat's what I do, at least @MiguelX413@pleroma.miguelcr.me
- panic 3y agoAccording to this page, there are 20k+ active nodes on the fediverse: https://fediverse.observer/stats https://fediverse.observer/stats
- anonymous_sorry 3y agoThanks for this link! I wonder why France and Germany have 6 and 8 times as many servers as the UK. Mind you, per capita France and Germany also have way more servers than the USA. Interesting.
- dumpsterlid 3y ago[dead]
- ipsum2 3y agoHetzner is in Germany, not sure about France.
- byhemechi 3y agomy guess is OVH
- input_sh 3y agoFor one, Mastodon's author is from Germany, but I don't think that has a lot of impact. What definitely does is that the major European server providers tend to be located in one of those two countries (like OVH or Hetzner). Note that server location != user location, users could be anywhere in Europe without a noticeable latency. Doesn't make any sense for me to pay for a server located in my country when I can find one in Germany with an order of magnitude better performance for the price.
- tim_apple__ 3y agoWhat do you think of Keet?
- pfraze 3y agoI haven't checked in on them lately but it's by the crew I worked with on Dat and Beaker. I tried the early demo and thought they did a great job with the application, and I know they've continued to make break-throughs on the hole-punching tech (in fact they rebranded to Holepunch). They're really talented folks so if you have any interested in p2p comms, I'd give them a look. https://keet.io https://keet.io
- Randor 3y agoWould it be correct to call Bluesky a "federated multi-cloud"?
- pfraze 3y agoI'm really not sure. I talked a bit at the end of the post about my ambivalence about the term federation for the networking model.
- Randor 3y agoNo worries. We faced some of the the same choices when we were designing the Windows 10 Delivery Optimization service. We initially based it off the Bittorrent protocol with a modified BEP30. I think MSDN just says "Cloud Managed". ;)
- bjord 3y agoI wish there was a version of this that stooped to a slightly lower technical level, or linked elsewhere to where I might learn more. Given that the tech is relatively novel, are there many other friendly writeups like this?
- HillRat 3y agoThe actual specification (https://atproto.com/guides/overview https://atproto.com/guides/overview) is relatively compact and quite readable.
- apitman 3y agoI don't think we necessarily need a single protocol to "win", but I will say the Bluesky/ATProto folks seem the most pragmatic when compared to the ActivityPub and nostr communities.
- kevinak 3y agoI have a hard time seeing how Bluesky/ATProto is more pragmatic than Nostr. Can you elaborate?
- vegabook 3y agoI'm interested in this too. While I note the (slightly chaotic) plethora of NIPs[0], and many of them look blockchainy, NIP-01 looks ultra pragmatic, simple, and flexible for a bunch of possibilities, and is the only one that is required[1] to be implemented. [0] https://github.com/nostr-protocol/nips#list https://github.com/nostr-protocol/nips#list [1] https://nostr.com/the-protocol/nips https://nostr.com/the-protocol/nips
- deleted 3y ago[deleted]
- apitman 3y agoNote that I specifically was comparing communities, not protocols. I actually think ActivityPub is pretty solid, though it's also the one I have the most experience with. Two of my main grievances with the communities both have to do with identity: ActivityPub as implemented in the wild doesn't make it easy to migrate your identity between instances. This has caused endless problems and drama with instances shutting down. You also have to use multiple accounts if you want to interact with different formats, ie Mastodon vs Lemmy vs PeerTube. As a user that seems pretty ridiculous. You're losing a huge portion of the advantages of federation. IMO a better model would be for you to create a single account on a universal ActivityPub server that supports all data types. Apps like Mastodon would act as clients, not servers. My reading of the specs makes me think this was the original intention (there is a client to server "C2S" protocol), but then Mastodon got big and no one ended up implementing the C2S protocol, treating Mastodon as the de facto C2S API. Note there is some work to create these universal servers[0], but I wouldn't hold my breath on Mastodon/Lemmy/etc implementing the C2S protocol. As for nostr, private key management is a complete non-starter for identity, barring some major new breakthrough. DNS is the pragmatic solution to identity. [0]: https://codeberg.org/Vocata/vocata https://codeberg.org/Vocata/vocata
- rokkitmensch 3y agoJust to trot out the cynical devil's advocate for half a moment, a) because normies don't care and b) we want to own the namespacing.
- DeathArrow 3y agoWhat problem is Bluesky trying to solve?
- davidgerard 3y agoTrump being banned from Twitter. That was literally why Jack Dorsey seeded this thing and hired these particular ex-cryptocurrency guys. He wanted a network where people couldn't be completely banned. (Since then the network attracted a userbase that laughed Jack off the site for vaccine denialism, so lol)
- rsynnott 3y agoI mean, that seems in line with general free speech principles, really; you’re allowed say whatever you like, but also others are allowed point and say “that person is saying something stupid”. Part 2 here sometimes seems to upset some ‘free speech’ advocates, but it really is part and parcel of the whole thing.
- MomoXenosaga 3y agoYeah no. I remember when the US "accidentally" bombed Al Jazeera. Think it was Afghanistan but it could be Iraq. Whatever the point stands: Everyone loves free speech until somebody says something they don't like. Basic human behaviour.
- _heimdall 3y agoI think that's basic social behavior these days but I don't know if its basic human behavior. Maybe I'm just an outlier, but I honestly can't think of a time when I wanted to silence someone because I disagreed with their opinion. Say whatever you want, no one else should be able to stop that. And yes, unfortunately that opens the door for people to say things that a vast majority of us would disagree with to the point of thinking it dangerous. Unfortunately that's the price for free speech, drawing a line through a sea of gray area is an impossible task.
- mg 3y agoWhat I like about Bluesky is that your identity is a hostname. So you can just use your own domain for example. What I don't understand is why the bsky.social service (which holds most user accounts at the moment) doesn't redirect http requests to ones profile. Then you could just say "I'm https://gnod.bsky.social https://gnod.bsky.social on bluesky". Instead of "I'm https://bsky.app/profile/gnod.bsky.social https://bsky.app/profile/gnod.bsky.social on bluesky". Maybe they don't want to make bsky.social very convenient so people have more incentive to use their own domain. Or there is some other thought behind it. I don't know.
- pfraze 3y agoOh no that's a solid idea. It's just on the TODOs.
- JimDabell 3y agoIsn’t this the same thing classic OpenID did? And didn’t OpenID define delegation, so that you could give your own URL and still end up using a provider-specific login? Are they going to end up rediscovering all the problems and solutions OpenID did?
- YetAnotherNick 3y agoWhat problems did OpenID face. It definitely looks bit risky but I can't think of any concrete problem.
- noirscape 3y agoOpenID faced the problem of an underdefined spec, causing every OpenID provider to do things just different enough for them to not be portable with one another. Which is why OID withered away in favor of just implementing the biggest social media login used by your audience (Google, Microsoft, Twitter, Facebook and if your audience includes gamers, Steam) and relying on that instead.
- p4bl0 3y agoThe problem I see with this idea is that if people using the default xxx.bsky.social starts to effectively use it as a direct HTTPS link to their profile, it will largely de-incentivize to use your own domain (and thus be able to easily switch provider) because you would lose this behavior since the domain you use is already your personal web page or blog or whatever, it won't be a direct link to your Bluesky profile. A way around this issue would be for users to create a specific subdomain for their Bluesky handle (like @bsky.example.net instead of @example.net) but that defeats the purpose in term of "branding", shortness of the handle, etc.
- codemusings 3y agoI just love how Bluesky devs get hung up on technical details like this and nobody seems to be concerned in the slightest how to position Bluesky as an actual product in the current market let alone pay for it in the long run. The end user doesn't care if it's federated or peer-to-peer if you ultimately end up seeing Ads anyway and it's still controlled by a single group just like Twitter.
- davidgerard 3y agoand that the linked post is a lengthy apologia for the network having an incredibly obvious unremovable central point of control even as they pretend it doesn't. Centralisation is a feature in social networks: being able to find your friends is the very first thing. But don't then pretend you aren't. Every now and then I post to Bluesky asking: so what's the loadout for Bluesky? How many servers does it take to run this thing? What would it take to run my own? These questions never quite seem to get answered or documented that I could find. A "PDS" is a reasonable Ubuntu box. They seem unable to admit what it takes to run the rest, though.
- pfraze 3y agoSorry, can you explain what unremovable central point of control you're referring to?
- noirscape 3y agoLooking at the federation architecture[0] shows that the network is inherently reliant on using a relay (or BGS - basically it's a giant data gathering server that generates the "external timeline" for applications using ATproto) - it's not possible for two PDSes to directly connect with one another (since a PDS does not store data from external users, but a relay does), which means that if a large relay falls apart, the entire part of that network goes down with it. Similarly, if a relay decides to nullroute you, you are functionally up on shitcreek, which because it's all handled by the relay is way worse than getting a Mastodon instance filtered. If we use a torrent analogy, it's not possible to use Bluesky/ATproto with DHT/peer discovery, you're required to connect to a tracker who then decides who you can peer with. (Relatedly, AcitivityPub also supports the use of relays for discovery, but doesn't require them. It's just not often used because they tend to be a moderation firehose.) [0]: https://blueskyweb.xyz/blog/5-5-2023-federation-architecture https://blueskyweb.xyz/blog/5-5-2023-federation-architecture
- 1vuio0pswjnm7 3y ago"The indie hacker spirit was strong in the NodeJS & Web community in 2014. A couple of things then happened all at once: DJB's NaCl became widely available, and, with it, more compact public keys" NaCl was widely available in 2011. (Maybe as early as 2008 TBH.) What came in 2013 was that someone took djb's concise presentation of the code and added some cruft to make it palatable to a wider audience.
- mike_hearn 3y agoInteresting to see the footnote at the end about enterprise/consortium blockchain systems. I designed one of the more popular one of those. If you've got questions Paul feel free to email me. Naturally I'm very biased, but I wouldn't say they were all developed merely so companies could have a blockchain strategy. Certainly just keeping up with the Joneses was a motivation for some of the middle/upper executives who sponsored some of the apps built on those platforms, just like it is for AI now, and was for XML back in the day. But there were also actual real business problems worked out in detail, with people who genuinely wanted to solve them and saw a chance to use blockchain hype as a Schelling point to coordinate large numbers of companies to collaborate better. And real apps have been developed and shipped on these, that provide at business value. It'd probably be interesting to write about some of these systems from a purely technical perspective, as a decentralised database engine, because the underlying protocols and designs are pretty interesting and (at least in Corda's case) quite powerful too.
- neongreen 3y agoYes! This would be a super interesting post. Do you have a blog/twitter/bluesky (hah)/anything that I could use to get notified if you write anything on this topic?
- mike_hearn 3y agoI have https://blog.plan99.net/ https://blog.plan99.net/
- api 3y agoNAT and firewalls, the trillion dollar hacks that break the Internet and require massive investments in central systems to do what endpoints could simply do if they networked together. (Trillion dollar because that’s probably what they really cost in lost efficiency, at least.) Systems to defeat these exist: ICE/WebRTC, ZeroTier, Tailscale, dozens of other protocols I can’t recall, but they all add overhead and require coordination. There are also many of these systems which means there is no standard. There is no equivalent to TCP/IP that is universally available that works around the ugly hacks we have used to destroy the potential of the Internet. It would be doable if everything were a desktop or a laptop, but most people use casual apps like Bluesky on phones. Phones have huge limits mostly around battery life. An always on P2P app on a phone will keep radios from sleeping and reduce battery life up to 50%. It’s also a problem for people who have crummy metered mobile plans. Firewalls require keepalive every 15 seconds or so or traversal must be done all over again, which means continuous chatter just to keep peer to peer links available. If phones could just call each other without executing what amounts to a coordinated attack against middleboxes in the way, low power peer to peer might be feasible. You could do things like time synchronize so peers sync at designated times and the radios sleep the rest of the time. But we can’t have nice things. All the heterogeneity and complexity in the Bluesky protocol exists to work around this. Without NAT and firewalls all you would need is maybe directory servers to find people and something to do the equivalent of dynamic DNS for when IPs change. That’s it.
- pfraze 3y agoConnection establishment is only a tiny portion of the blockers that stopped us from using P2P, as I discussed in the post
- crimsoneer 3y agoI will say that Mastodon actually does this quite well... they key challenge is lack of US adoption, which means whole US centric interest groups (like AI/LLMs) aren't on it. But if you're in, for instance, German hacker spaces, it's a perfectly passable Twitter alternative with actual working federation.
- mikeravkine 3y agoThe challenge to Mastodon isn't adoption, its absense is a symptom of a deeper issue: it's a Very Bad Idea to trust random server operators with both your data and it's sovereignty. You can take your data to another server only if the original server is up. When the admin forgets to pay the bill and ISP nukes the box and the backups admin said they had don't exist and the coadmin turns out to have no access at all, you lose everything. Ask me how I know. Self hosting in theory should fix this but it doesn't: it's a resource intensive afterthought because the developers only really care about their Big Instance.
- RobotToaster 3y agoDoes bluesky even federate (or whatever they call it) yet? Last i saw it still relied entirely on a single central relay.
- realfeel78 3y agoNo. And it probably never will.
- pfraze 3y agoWe spent a lot of time delaying open release to finish building federation. If that doesn’t signal our commitment to it, I don’t know what will
- thejohnconway 3y agoIs there any plan for (or thinking about) an AT-ActivityPub bridge within Bluesky?
- xwowsersx 3y agoI had initially forgotten that I had registered for Bluesky and even had it installed on my phone. When I signed up, I was intrigued by the notion of something novel in this space. So the I launched the app the other day, only to find that it seemed like a less efficient and slower version of Twitter. I couldn't identify any hints or nods toward potential future product innovations or features that would justify my continued use of Bluesky...or anyone's for that matter. I'm likely overlooking some aspect of its functionality. Does anyone have more information or insights to share?
- rdlw 3y agoI think the main selling point is that it's a slower version of Twitter. I'm not sure what you mean by "less efficient" but I definitely know I don't want Twitter to do whatever it does more efficiently.
- xwowsersx 3y agoHaha well you might not want it to, but if you're a user of such a service, you probably do want it to do what you're there for efficiently
- rakoo 3y agoSo, if I want to be uncharitable, you have PDS servers which are basically web servers with a signed Atom feed, and "Big Graph Servers" which are basically search engines indexing only those servers and giving their own algorithms, right ? Couldn't we make this with existing tech ? Having your own user id as your own domain is easy when everyone is on Bluesky but can already be done today. What's missing ?