10 ms·
Copying over my latest backend status update; figure folks would find it interesting Servers are holding up so far! Fortunately we were overprovisioned. If we
by pfraze 2y ago
Copying over my latest backend status update; figure folks would find it interesting
Servers are holding up so far! Fortunately we were overprovisioned. If we hit 4mm new signups then things should get interesting. We did have some degradations (user handles entering an invalid state, event-stream crashed a couple times, algo crashed a couple times, image servers hit bad latencies) but we managed to avoid a full outage.
We use an event-sourcing model which is: K/V database for primary storage (actually sqlite), into a golang event stream, then into scylladb for computed views. Various separate services for search, algorithms, and images. Hybrid on-prem & cloud. There are ~20 of the k/v servers, 1 event-stream, 2 scylla clusters (I believe).
The event-stream crash would cause the application to stop making progress on ingesting events, but we still got the writes, so you'd see eg likes failing to increment the counter but then magically taking effect 60 seconds later. Since the scylla cluster and the KV stores stayed online, we avoided a full outage.
- delichon 2y agoDoes Bluesky intend to be responsive to the kind of court orders that X rejected?
- mwilcox 2y agoClearly given he is excited to receive the traffic
- meiraleal 2y agoInterestingly, the cohort moving to Bluesky isn't the same as those who were having trouble with the law.
- siproprio 2y agoYes. Brazilians were censored and moved to bluesky. Elon musk has problems with the law! Absolutely no overlap whatsoever. Also very interesting in its own way.
- jrflowers 2y agoCan you elaborate on this? I’m not clear what you mean here.
- pfraze 2y agoSpeaking entirely personally as I don't handle those questions at bsky. I couldn't even begin to comment without seeing knowing what the court orders were and what the cases are. Every social company operating internationally runs into this issue, and it's daunting to say the least. So, again, this is not something I decide. What I can say is that the protocol is a neutral global layer for data, which can then enable multiple applications with their own moderation policies and decisions. There's always going to be moderation decisions we make that people will disagree with. The point is that something can be done about that disagreement - you can have other applications on the same network that makes their own decisions. I think one of the best things that could happen is that Brazilian developers fork the Bluesky app and build locally-owned social platforms on the atmosphere.
- Kye 2y agoContext: atmosphere is like "Fediverse" but for the AT protocol and also way more schway
- chrisweekly 2y agoI had to check, and urbandictionary confirms, "schway" is > "A word to use when something is astonishingly cool, fashionable, or popular."
- Kye 2y agoI got it from Batman Beyond.
- throwaway48476 2y ago>Brazilian developers fork the Bluesky app and build locally-owned social platforms on the atmosphere. This makes it easy to send thugs to their house and 'encourage' them to comply with any political demand. Jurisdictional arbitrage, particularly sites hosted in America following the first amendment is the only thing keeping anti authoritarian speech alive in authoritarian regimes. https://www.dropsitenews.com/p/audio-pakistan-global-critics https://www.dropsitenews.com/p/audio-pakistan-global-critics
- mensetmanusman 2y agoNo, it is a game of whack a mole.
- deleted 2y ago[deleted]
- crossroadsguy 2y agoSomething tells me at the least bsky will not begin a name calling primary school attack with a supreme court justice of a sovereign nation while using abusive names for the president of that country all the while using childish fake AI generated graphics. PS. Somehow all of this feels very natural when done by Musk. It kind of makes sense. It feels like “yes yes, he’d do this”.
- rapsey 2y agoA supreme court judge clearly acting against the laws of their state, but somehow Musk is the bad guy.
- bilvar 2y agoWe arrived at the point that people are celebrating censorship and authoritarianism because of their irrational hatred of one guy.
- Ygg2 2y agoThat's how it always works. "You see the problem with not dealing with XYZ is that the government/judicial/legislative doesn't have enough power to tackle XYZ." You grant them power they asked. They don't deal with it, but ask for more power. At some point they stop asking and label you as the problem.
- rapsey 2y ago[flagged]
- louhike 2y agoThat’s interesting. Why do you use event sourcing? Is having a full history important for a website/app like bluesky?
- pfraze 2y agoAhhh you know what, I should call it stream processing or something, because we don't store the data entirely as events. We store the data as a mutable K/V which emits an event stream of changes, which can then be ingested into different views. We chose not to store changes as events specifically because we don't want unbounded growth in the system. Initial syncs work by fetching the current state of the K/V store (the "data repo"). Bluesky is built on atprotocol (atproto.com) and can be thought of as an open distributed system. The event stream is for replicating throughout the various services.
- geekodour 2y agohi pfraze, can u tell us a bit more about the golang event stream? does it also trigger the computation of the views periodically? more precisely wanted to understand how do you generate the event stream from sqlite
- pfraze 2y agoIn my previous description, I avoided talking about atproto details for clarity, but this is all part of that (atproto.com). The "kv stores" are what we call data repos[1] and they use sqlite for storage, but can produce individual event streams. Those streams flow into the golang event stream, aka the "relay". View computation happens continuously. 1. data repos are actually signed merkle trees, which gives at-rest authentication of the data as it gets shipped across organizational boundaries
- pcwalton 2y agoIt's frustrating that anything related to X/Twitter is such a predictably-partisan tinderbox because this is really interesting technical information. Thank you for sharing it!
- kstenerud 2y agoIt's partisan/political because Musk is partisan/political. And it's not just Musk. We've been living in a fantasy land of "no political affiliation" in the tech world for decades, and now that the age of the hyper-rich has come once again, they are realizing the benefits of using the power they wield to shape the worlds they live in. So now in the early stages of this century's great fight, we'll see our beloved tech giants join the political fray in full force, dragging their follower armies along for the ride. And it works, too. Just look at the comments here.
- hereme888 2y agoyup, just read the comment.
- Dalewyn 2y ago>once again When has the rich not been more or less 1:1 with The Powers That Be(tm)? The form changes with the sands of time, but the essence is always the same.
- roenxi 2y ago> We've been living in a fantasy land of "no political affiliation" in the tech world for decades Which "we" are you speaking for? That doesn't sound like the tech world, we've had a lot of explicitly political tech movements over the years. Off the top of my head some of the more successful and major ones were: - Cryptography. - Free Software. - Cryptocurrency. The Silicon Valley based companies have been politically active since at least 2016, there have been a couple of political exoduses by various groups to alternative platforms. The non-Silicon-Valley companies have been worse and generally suspect to the point where nobody expected political apathy (is anyone going to claim that Chinese social media are not politically subordinate to the state?).
- johnchristopher 2y agoThanks for the details ! I wonder about: > Hybrid on-prem & cloud. I wonder about the factors/considerations that led to hosting a given data/services on-prem or outside. Were there purely technical considerations or were there also about "self host what can't get leaked" (think GDPR, privacy concerns, things like that) ?
- Angostura 2y agoI'm just trying to work out why I'm seeing so many posts in Portguese when all my settings are set to English
- johnisgood 2y agoHmm, it is something I would use Elixir for.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]