15 ms·
The inside story of Threads
- colesantiago 3y ago> Instagram uses Python (Django) for its backend. By using the same backend for Threads, we could leverage a lot of the existing tech stack for Threads and reuse most of our existing data models, business logic, security features, and server infrastructure. This also meant users could sign in to the app with their existing Instagram account, making it super simple to onboard and set up your Threads app. Looks like Python and Django is extremely scalable at Facebook scale, even after using Threads I didn't realise any downtime at all. It would be possible for Mastodon to learn from this so that they too can scale to 200 million users.
- organsnyder 3y agoMastodon has such a different system architecture—at a business level—that I think many of the scaling challenges are entirely different. While my experience with Mastodon is as a (very satisfied) user, in general scaling challenges are as much figuring out how to scale the interactions between systems as they are scaling the individual systems themselves. That gets incredibly more difficult when those individual systems are spread out among various organizations with different priorities, philosophies, technical abilities, and resources.
- YouSuck2 3y ago[flagged]
- weego 3y agoThe only lesson to learn from this is that given exceptional resources and a long list of talent any stack can be made to scale. That doesn't help most projects in the world.
- tjrgergw 3y ago[flagged]
- roflyear 3y agoPython apps using popular frameworks (not all frameworks) are generally (not always) easy to scale. The app itself is mostly just "spawn another instance somewhere" - storage and sessions and all that can be complex, but that's not really impacted by the app framework/language.
- tivert 3y ago> The only lesson to learn from this is that given exceptional resources and a long list of talent any stack can be made to scale. Exactly. IIRC, Facebook itself is/was a giant ball of PHP, and they had to do all kinds of crazy things to make it scale, up to and including writing their own compilers and VMs. That doesn't say anything about the suitability of the scalability of stock PHP.
- ozzydave 3y agoI think the lesson is, avoid rewrites and don’t break apart your monorepo; have teams improve the platform while your product engineers go full-steam ahead. Compare with Twitter which spent 2 years in the woods rewriting to Java while FB kept growing, giant ball of PHP and all.
- highwaylights 3y agoYou’re getting downvoted for this, but you’re not wrong. Just keep moving forward unless the sky is falling is 100% the lesson to be learned here.
- esafak 3y agoIn the worst case, you'll fail to scale due to technical shortcomings like Friendster and become forgotten while later entrants like Facebook eat your lunch. There was a time when "being web scale" was a real concern. That's one reason companies like Google make such a big deal about it in their technical interviews. Now most of the hard work has been done and exposed as services so today even an untrained teenager can stand up a web-scale service.
- willio58 3y agoI don't use Python or Django anymore but I'm always happy to hear about it and kinda surprised to learn it's still being chosen today for massive-scale social media sites. To me, Django was my go-to for cranking out a usable MVP in the shortest time possible. It made it so easy, I could define a DB schema and it would generate these forms with validation built in. I kinda miss those days.
- roflyear 3y agoWhat you're saying is correct, but oh boy have I worked on Django apps that have had awful security issues because of this. "look, all my endpoints are there, user A must only be able to reset their own password... right? ;)" Also you don't have to miss those days! You can still use Django or a similar library? I still use Flask, it is great.
- willio58 3y agoI'm pretty deep in the Typescript world these days but hey, maybe one day I'll make my way back into Python!
- colesantiago 3y agoI sort of wish that Go, Rust, Zig has these kind of frameworks like Django, Laravel and Rails. I believe there is an opportunity here to make the next Django in these languages.
- olah_1 3y agoEach of those absolutely do have them. Here is a list for just Golang: https://github.com/avelino/awesome-go#web-frameworks https://github.com/avelino/awesome-go#web-frameworks
- ajhai 3y agoThere are a lot of things that goes on in production to scale any service to handle that level of requests. But Django as a web framework is good at what it does. It comes in with most things that one needs to put web apps in production. We recently open-sourced an LLM apps platform (https://github.com/trypromptly/LLMStack https://github.com/trypromptly/LLMStack) that is entirely built with django as backend (drf for APIs, channels for websockets and reactjs for frontend).
- Mxbonn 3y agoThis is largely due to Meta maintaining an internal performance-oriented production version of CPython: https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder .
- thewataccount 3y ago(I use django in production myself, nothing against it) It's probably worth noting Meta made their own JIT'd version of CPython just for Instagram/Threads. Maybe mastodon devs can use their interpreter, but Meta's human/compute resources are magnitudes more than mastodon, the lessons from Meta might not be very applicable. https://github.com/facebookincubator/cinder https://github.com/facebookincubator/cinder
- ushakov 3y agoIsn’t Mastodon written in Ruby?
- thewataccount 3y agoYes, I wasn't sure if parent was suggesting they switch to django, or simply "using what facebook learned" - I didn't want to imply that there's no way Mastodon devs couldn't use Cinder to their advantage. I do however highly doubt Cinder would be of use to Mastodon at the current moment.
- parthokr 3y agorecently they made cinder public… this is a JIT’d version of cpython
- smfugit 3y agoThese systems should not scale. Giving everyone free broadcast capability is a sign of mentally bankruptcy. Its like giving every neuron in your head broadcast capability. What happens when they all fire at the same time? Or like giving everyone in the world a radio transmitter that can broadcast on any frequency. What happens when they all transmit on the same frequency at the same time? These idiots are scaling things because its become possible technically. There is no other logic to whats going on. And its producing/enabling massive amounts of useless noise. The UN report on the Attention Economy says 0.5% of content on the net is actually consumed by anyone.
- tivert 3y ago> These systems should not scale. Giving everyone free broadcast capability is a sign of mentally bankruptcy. ... These idiots are scaling things because its become possible technically. There is no other logic to whats going on. I agree, but I don't think it's a sign of "mentally bankruptcy," but an indication of the flaws and limitations of contemporary ideology. They're not just doing it because it's technically possible, they wouldn't even try and no one would sign up if our ideology didn't say it was a good thing to do.
- deleted 3y ago[deleted]
- freedomben 3y agoPython/Django (and most other frameworks) isn't the problem with scaling. It's following the 12-factor app rules.
- bee_rider 3y agoPython is known to be pretty wasteful in terms of compute performance, but is the compute grunt of a given node a big deal in these kinds of networks? I assumed it is a big communication problem. Python shouldn’t hurt there, right? It isn’t like a Python-generated packet goes through the network more slowly.
- KaiserPro 3y ago> t would be possible for Mastodon to learn from this so that they too can scale to 200 million users. I mean possibly, if it was designed to be a centralised bohemoth with a staff of 600 to look after it. Its decentralised by design, much like the web used to be.
- interactivecode 3y agoSo did they setup a separate backend for threads or did they just extend the current instagram backend to support text posts?
- echelon 3y agoDoes anyone have DAU/MAU or "post" numbers for Twitter, Threads, Bluesky, and Mastodon? Even if only estimates? Who's growing fastest? Who's stagnating?
- burkaman 3y agoMastodon stats: https://fediverse.observer/stats https://fediverse.observer/stats I haven't found public stats for any of the others.
- IG_Semmelweiss 3y agoWow. Can we infer from the precipitous dip in MAUs in the 1st half of the year that half of the userbase for Mastodon is under 21 (busy with schoolwork?)
- mzs 3y agoYou can get a sense for bluesky from the AT firehose: https://firesky.tv/ https://firesky.tv/
- mandeepj 3y agoNoticed Threads during the second launch; it was earlier released during 2019. I forgot there's an app called Threads. I bet if an average person like me launched it, no one might have even noticed it. Such is life.
- DueDilligence 3y ago[dead]
- jshchnz 3y agoIf you're looking for a deep dive into the Threads apps, can check out this non-Meta written post: https://www.emergetools.com/deep-dives/threads https://www.emergetools.com/deep-dives/threads
- JimDabell 3y agoAlso some more details here, interviewing two team members: https://newsletter.pragmaticengineer.com/p/building-the-threads-app https://newsletter.pragmaticengineer.com/p/building-the-thre...
- miguelazo 3y agoI'm baffled by the amount of interest in Threads, but I guess some of the technical stuff is novel? I don't know a single person who uses it.
- jedberg 3y agoThe technical feat was impressive. 100 million users at launch with no major downtime. Most people here are interested in that story.
- miguelazo 3y agoIs that really impressive at this point? With all of the frameworks, cloud infrastructure, etc? And the resources of one of the biggest companies in the country? What’s impressive is that Meta would waste resources on this when their Oculus division is already such a disaster.
- kenferry 3y agoServices with network effects go clique by clique. You don’t know anyone on it. For others, their entire social circle uses it.
- nebula8804 3y agoYeah the flamewars between Mcdonalds and Wendy's accounts are epic. I'm sure they will be longtime Threads users. Unfortunately for my cadre of human based contacts, it is a complete desert. Everyone was intrigued for a week or so, some tried to keep it going a month later but we even have people that would benefit from leaving Twitter like AOC having abandoned the platform.
- jedberg 3y agoSay what you will about Meta and their products, but I have mad respect for their production engineering team. Keeping things like Facebook and Instagram running is no easy task. You can't just cache your way out of it like most apps. People expect to see their friends' updates instantly. I've been super impressed with the technology part of the Threads launch. I can't wait to read more about it in the future.
- gtowey 3y agoAgreed. There are a lot of really smart engineers at these companies. No doubt about it! They often have to pioneer solutions to challenging problems, which have downstream benefits for everyone else. I do wonder though if we will look back on this period in history and regret the economic and cultural incentives which have co-opted so many smart people into the service of ... advertising.
- mdgrech23 3y ago100%. We spend more money working on cures for erectile dysfunction than ebola. We spend more money working on cures male pattern baldness than we do diseases that plague the third world. How many smart amazing engineers have worked for the auto industry over the last 100 years when their expertise could of been used in a civil manner to improve infrastructure and our overall existence? This is capitalism and whoever has the money can buy up the most talent. Whether that is a good use for that talent doesn't seem to matter.
- csomar 3y agoI am guessing the people working on the erectile dysfunction medicines are motivated by financial compensation; and that it happens that the people who provide such compensation do happen to intersect with the people who have ED but do not intersect with those who have Ebola. This is more than capitalism, it's freedom. If you are suggesting that we spend more money on Ebola or public transport, that's tyranny.
- 3y ago
- Yusefmosiah 3y agoThey didn’t use React Native, but wrote separate iOS and Android apps. Does this mean that React Native has lost support and investment from Meta?
- drak0n1c 3y agoWith SwiftUI and Kotlin Jetpack Compose you can write architecturally and syntactically similar code across the two platforms. Essentially the same compact classes with the differences just being view property differences and OS API discrepancies (location, etc). So it's become a lot easier to manage separate development while keeping design direction and architecture on the same page. This is more so the case for apps with simple aesthetics and simple user interactions like Threads. It's not necessarily a sign that they are dropping support for React Native.
- rollcat 3y agoWell put. I intended to reply with something like "keep simple things simple" but you gave it much more depth.
- KaiserPro 3y agoIt means you get many more promotion points for creating a new framework and shipping it in a new product. Also, generally internal tools are exceedingly poorly documented and supported so it seems easier to write your things from scratch.
- JimDabell 3y agoAs the article states, they reused a tonne of things from Instagram.
- jamil7 3y agoI don’t think they’ve ever really built entire apps in React Native outside of maybe messenger? Usually a few screens are implemented in it and the rest native. Or maybe features are prototyped in it and later rewritten.
- sentimentscan 3y agoStill not available in EU, as they will not allow to use email instead of instagram accounts [1] 1. https://www.theverge.com/23789754/threads-meta-twitter-eu-dma-digital-markets https://www.theverge.com/23789754/threads-meta-twitter-eu-dm...
- paulddraper 3y agoGeez what an oversight
- johannes1234321 3y agoIt's not an oversight. Let's them start with less hurdle from oversight authorities. Also focussing on a market reduces investment in localisation and then having unsustainable small local bubbles. (And ensures I won't waste my time trying it, as I'd have to bypass too many hurdles for making it work)
- paulddraper 3y agoBut isn't the appeal of the Twitter is a global public square?
- johannes1234321 3y agoThe "global square" isn't a single thing, it is an intersection between many many small sections in a bazaar. If my peer group isn't there I run away quickly. The newest outrage in US politics is only of limited interest to international audience, but they "need" the local outrage. As is the Baseball "World Series" maybe relevant in the U.S. but outside barely anybody cares. For being successful you have to get the local number one stars on the platform. Good coverage of the favorite sport and sports teams. And a group of locals to discuss those things. Sure American culture has some relevance globally, but over large parts even Europe itself is quite heterogenous. Once you got it big enough it runs with less oversight. But mind: you can't afford too small growth being Facebook/Instagram/Meta and competing with Twitter/X for people who haven't left already as well as BlueSky, Mastodon, ...for people who already left sets a high barrier. Just see all the news about people leaving Threads already (which is to be expected - people try it out, see nothing, leave, a subset stays and then maybe grows it)
- yieldcrv 3y agoI don't see this really making people care about the Fediverse anything I can read or watch about federated social media trends?
- senkora 3y agoThe best way to stay in the loop on federated social media trends is to join a fediverse instance. It gets discussed a lot there. Anecdotally, there was a lot of discussion within the fediverse when threads was announced and the general sentiment that a lot of people had, was that instances should immediately defederate from threads to avoid eternal september. Personally, I argued for staying federated with them because I think that would lead to a healthier fediverse. What I envision is: 1. Most normal individuals will be on a corporate-backed, ad-supported instance like threads. 2. Companies and governments will be on custom-domain instances managed by a fediverse service provider who they pay directly, both for branding and in order to avoid being beholden to the poor customer service of the corporate-backed instances. 3. Some tech-savvy individuals will be on small or self-hosted independent instances that provide a differentiated experience in some way. (This is most of the current fediverse) This is basically how email works now and so it seems like the most likely end-state.
- styfle 3y agoAre any of the big email tools planning to add support for ActivityPub? They already have your corp domain so it feels natural. For example, I could see a future where MS Exchange has the ability to federate with Mastodon and optionally enable/disable employees through ActiveDirectory, etc.
- davidw 3y agoI've been enjoying Threads. What with Musk going all in blaming the ADL for his problems... that's just too much. The guy is unhinged. I used to think of him as quirky or eccentric and appreciate some of the good things he's been involved with, but he's gone off the rails. Threads is still missing some features, and there's less traffic, but it's probably healthier for me to spend less time on social media anyway.
- deleted 3y ago[deleted]
- ar_lan 3y agoMusk is definitely unhinged on his ADL attacks, but the ADL has some... strange takes. They have whole articles on how the Bowl Cut and the OK symbol are hate symbols. For the OK symbol they have nested "disclaimers" (why these are not prominent right from the get-go is beyond me, however). And the bowl cut page doesn't even hold any disclaimers at all - as if the 5 year old whose Mom doesn't know how to give a haircut is really very dangerous? * https://www.adl.org/resources/hate-symbol/bowlcutdylann-roof https://www.adl.org/resources/hate-symbol/bowlcutdylann-roof * https://www.adl.org/resources/hate-symbol/okay-hand-gesture https://www.adl.org/resources/hate-symbol/okay-hand-gesture
- davidw 3y agoThe okay thing is pretty real. I worked for a company that changed its logo so as to avoid the one they had, that was an 'okay' gesture. They were big enough that that decision cost them time and money - they didn't make it lightly. With bowl cuts, they're not talking about 5 year olds and their moms, obviously, but grown-ass men who know better. All people are susceptible to 'patternicity' - seeing a pattern where there is none, and it's possible they blow a few things. But that kind of pales with him singling out the Jewish group for attention and blame.
- ahahahahah 3y agoDid you read that bowl cuts article? Or just spreading more alt right nonsense without any thought of your own? You are completely misrepresenting what that article about bowl cuts is about.
- lopkeny12ko 3y ago> With everything moving ahead at full speed toward a July launch, our team saw and took an opportunity to release the app early. On the upside, this meant Meta would be able to offer a new social app at a time when people were eager for a new experience. Wow, this seems like an unnecessary dig at X.
- constantly 3y agoHow so? Aside from the name change X was exactly the same experience that Twitter had been giving for a decade or more.
- Me1000 3y agoNot "exactly". Twitter artificially boosted its new owner's tweets, the site regularly returned errors after shutting down data centers, and users who were previously banned for hate speech were allowed back on the platform. And while 3rd party apps might not have had huge adoption relative to the size of twitter, I know a few people for whom that was the last straw.
- deleted 3y ago[deleted]
- pictur 3y ago[flagged]
- smcin 3y ago> Five days [after July 5], over 100 million people had joined Threads, making it the most successful app launch of all time. Err, no. The 100m were mostly migrated from existing Instagram users (which has ~500m DAU/2.35b MAU), not new joiners. Gizmodo [0]: "[one month after launch] Threads Has Lost More Than 80% of Its Daily Active Users... DAU down from a peak of 44m to 8m, and falling at about 1% every day. Engagement was down to just 2.9 minutes a day compared to 19 minutes earlier in July." If 100m existing users who briefly came over from Instagram and then mostly became inactive is "the most successful app launch of all time", then by that metric was Google+ forcibly converting new YouTube or Gmail users/commenters the most successful website-based [social] launch of all time? [1] "1/2015: There are about 2.2 billion Google+ profiles. Of these, about 9% have any publicly posted content. Of those, about 37% have comments on YouTube videos as their most recent activity, and another 8% have profile-photo changes." Can anyone cite third-party DAU and MAU stats on Threads, currently? Or how many Threads users quit Twitter, and have remained active on Threads? [0]: https://gizmodo.com/threads-has-lost-more-than-80-of-daily-active-users-1850707329 https://gizmodo.com/threads-has-lost-more-than-80-of-daily-a... [1]: "Nobody is using Google+" BusinessInsider 1/20/2015 https://www.businessinsider.com/google-active-users-2015-1 https://www.businessinsider.com/google-active-users-2015-1
- hbn 3y agoTo be fair, users did have to install an app and go through the onboarding to be counted. That is a big ask for most users! That wasn't the case for Google+ where they straight up just considered all Google accounts to be Google+ accounts suddenly. And that was on the web where you don't have to install anything. But I don't disagree, saying they got a lot of signups doesn't mean it's a home run success. Myself and most of the people I follow on IG signed up within the first few days, made a post that said something along the lines of "what's a Threads" and then haven't posted since then.
- ahahahahah 3y agoIf my app uses Google login, an I not allowed to count those users because I simplified their sign up flow too much? Your stance on this basically makes no sense.
- thorum 3y agoThreads was a breath of fresh air at first and I followed a bunch of people I already followed on Instagram and Twitter. Opening the app today there is exactly 1 post from someone I follow and pages upon pages of ‘recommended’ content from people I don’t care about, much of which is outrage bait and political content. Talking to other people I know this experience seems to be common. I understand the idea of wanting to make the app feel alive on day one by including content from popular influencers, but I genuinely think they would have had better retention if they went the other way and only showed you what you wanted to see - or at least separated the recommended posts onto another tab and gave better controls for filtering it.
- sidcool 3y agoI would love to know the entire backend stack of threads. From their load balancers to Database schemas.
- kridsdale3 3y agoApply for a job there, and you'll quickly learn.
- nkg 3y agoThey are kind of saying that they have become such a behemoth, with so much bureaucracy and inefficiency, that the only way they could move so fast is by building a sub team. Kudos for being honest.