12 ms·
How difficult can a platform like Twitter be? I am convinced you can run the entire tech stack with a team of a 100 people.
by holoduke 4y ago
How difficult can a platform like Twitter be? I am convinced you can run the entire tech stack with a team of a 100 people.
- kakwa_ 4y agoThat's likely optimistic. Maybe the user facing site, but that's just the tip of the iceberg. There are plenty of internal/backend/restricted systems to support and/or monetize this part. And that's not counting the huge number of support people & moderators needed.
- RivieraKid 4y agoFrontend, Android app and iOS app should be doable by 1 person each. Then one graphic designer and a project manager for all of the 3 frontends.
- benj111 4y agoWell graphics design isn't hard. I'm sure one of the Devs could do that. And I'm sure there are tools to convert an iOS app to/from Android, so one dev can do both. And a website is easy. You could do it with 1 person. But Elon is such a machine, he could keep it running by himself.
- RivieraKid 4y agoWell that's what I did, I made both iOS and Android version of an app and some of the design. I mean, a mediocre complexity app doesn't need more than one developer for a "slow improvement mode".
- benj111 4y agoBut Twitter needs more than 5 employees. How many bug reports do you think the Twitter site and apps get each day? How many employees do you think it takes just to triage them?
- RivieraKid 4y agoYou don't have to read all of the bug reports, just a representative sample to discover issues worth fixing.
- benj111 4y agoHow do you get a 'representative' sample? You still have to read them to work out that user X saying they can't log in is the same as user Y saying authentication is failing at A stage, issue is present on B and C platforms version D.E Further the issue of 'site using the wrong font' is not equally as important as 'site is exposing private user data'. And you're likely to get many people reporting the former, and maybe only one reporting the latter, so a sample isn't likely to catch the really important stuff.
- egman_ekki 4y agoHow many people would you say it would need for the backend?
- neongreen 4y agoNot sure if you’re joking or not, but that’s exactly how Telegram stems to work. One person per app (ok, for the desktop app it’s three people now).
- matwood 4y agoI'm all about lean teams, but this is a joke.
- megax 4y agoWho is defining and managing the PRDs and the roadmap in this scenario? What about user testing and customer development? By “graphic design”, do you mean User Interface design? If so, who is responsible for the user experience beyond just UI? The point I’m trying to make is that it takes some effort (beyond just the plumbing) to create an experience that folks actually want to use on an ongoing basis
- unionpivo 4y agoWe are not talking future development, just maintaining status quo. Rode map is just: keep it working, or whatever EM twitted about hour ago :) But I agree it's way to optimistic number. You need at least a few for each platform, just because of bus factor, you also need people to keep in touch with Apple/Google reps etc., making sure bills are paid etc. I imagine it will mostly be just minor tweaks and no major features, you can easily do both mobile targets by cca 10 people, not working crazy hours. There are plenty of successful apps with smaller teams, that make it work.
- pixl97 4y ago>There are plenty of successful apps Eh, tossing the direct comparison to Twitter out here because they seem to have gone too far the other way, this is much like saying "I can make 1,000,000 screws a day, so assembling at least 100 cars a day should be easy".
- unionpivo 4y agoThe hard part of twitter is on server side. I am not saying that writing twitter like app is easy, but writing it and keeping it running are two different things. If you are not adding new features, you are just mostly keeping up with Android/Apple platforms, which can be annoying, but not that difficult.
- RivieraKid 4y agoI'm maintaining status quo for an Android/iOS app and it takes perhaps 5% of my work hours. > also need people to keep in touch with Apple/Google reps etc. You certainly don't need people specifically for this task.
- donkeyd 4y agoYou're joking, right?
- lightbendover 4y agoShe or he is just naive. It does not take much distinction to post on HN.
- RivieraKid 4y agoI'm serious. Why would you need more than one developer for the Android or iOS app?
- caporaltito 4y agoFor the ad platform only, this would require already a hundred of people. Building an SSP is not an easy task. You are right about being optimistic.
- unlikelytomato 4y agoas someone who worked at an SSP, that is nowhere near what is needed for an established platform. There were a handful of engineers(10 if we are being generous including data science) keeping the lights on. The rest of the time was spent trying to prevent "senior" engineers from pushing code attempting to execute a sql query per each of the billions and billions of daily api calls.
- peoplefromibiza 4y agoWhatsApp, 50 engineers, > 1 billion users daily, sold to Facebook for $19 billions.
- threeseed 4y agoWhatsApp didn't have advertising, recommendations, bots or had to provide tooling for governments, regulators, content moderators etc. It is so incomparable in scope I don't know why people bring it up.
- Amezarak 4y ago> WhatsApp didn't have advertising, Is there a reason to integrating ads into WhatsApp would require more than another 50 people? Twitter ads are certainly do not appear very complicated. The most complicated thing about Twitter is scale, which is why the comparison is made with WhatsApp. > recommendations, Does Twitter have recommendations? From what I understand, the front page was actively curated - that is, a human chose stories to put there. I guess you could count the god-awful default feed ordering as "recommendations", but there is nothing advanced about it. > bots If WhatsApp doesn't have bots, it's the only social media/chat app I've ever heard of that doesn't. What is needed for this other an an API? > had to provide tooling for governments, regulators, content moderators etc. I'm sure at least some of this exists for WhatsApp. Nevertheless, how many additional employees does this have take? I am not sure why there is so pushback against the idea most companies are overstaffed. For the most part, yes, everyone has "work" to do. But most of the work is fundamentally unproductive. It's this way throughout the economy, but a few tech companies probably do represent extreme cases. I think the best argument for their case is that most of them are very profitable anyway (not Twitter, somehow), and they might as well throw money at thousands of people to do stuff in case one of them accidentally does something that ends up being wildly profitable. I am fairly neutral on the whole thing; I strongly dislike Elon, but I also think Twitter was horrifically mismanaged. While I doubt Twitter will come out better than it is, the idea that firing most of such a large organization would necessarily result in the immediate collapse of a mature product does not say much about the people that were fired. I'm more sympathetic to the idea that it would get even worse over time, but I don't think there's anything necessary about this. You could focus on resolving longstanding issues while pausing most new work and probably come out perfectly fine.
- objclxt 4y ago> I am convinced you can run the entire tech stack with a team of a 100 people. This is because you don't see the complexity. What you see as a Twitter user is a fraction of what's actually there. You have to build a platform for ads. Not just serving ads, but allowing advertisers to prepare their collateral, preview them, get their results, and be billed. So that's an entire content and invoicing platform separate from your main feed. And since your platform is all user generated content, you've got to build a moderation pipeline. A place for users to make reports, but also an interface for your content moderators to view content and make decisions. Oh, and while you're there you'd better build a portal for law enforcement to make data requests, along with your DMCA takedowns. Oh yeah, DMCA - that's another whole thing you've got to worry about. Then the EU comes along and needs you to build something to support your GDPR obligations. Then India wants something similar, but only for its citizens. Your users also want verification, so better build that platform for securely verifying accounts and awarding checkmarks. It snowballs. Was Twitter's engineering group bloated? Probably. Most large companies are. Could you run the whole Twitter tech stack as it exists today with a hundred people? Absolutely not.
- x0x0 4y agoAds isn't just serving, but targeting. The better you target, the more your ads are worth. When you have $5B of advertising, a 0.01% improvement is breakeven for $500k of fully-loaded comp. So you should add as many engineers / data scientists / etc as can generate an 0.01% annual improvement. Or maybe you want to take 3x their annual salary: that's still an 0.03% improvement in ad relevance. Separately, some commenters here are flatly delusional about the effort to ship a site, android and ios apps, internal mod tools, help docs, support, and legal docs in 34 supported languages. Not to mention obeying laws in all the countries that implies. Or image and video hosting! With recoding of videos, resizing of images, and the management of what is surely petabytes of images and videos with very high reliability! That is not a 1, 2, or 3 person job to do well.
- whywhywhywhy 4y ago>Ads isn't just serving, but targeting. Anyone who has used Twitter, have you seen any evidence they do this beyond extremely basic geographical targeting. Like people keep listing off all this stuff when we’ve all used the site and can see if it does have a team working on it then they’re not doing it to the levels of their competitors.
- cdf 4y agoGetting it to about 90% working in a single country or just the general western sphere, probably 100 will suffice. Making it globally available and legally compliant, that's where the next few thousand folks come in.
- BeefWellington 4y agoIndeed. Just translating it could be a team of at least N+0.5 people, where N is the number of languages you want to translate your product to (the +0.5 accounting for translation verifications). Even if you outsource it that just slows down feature deployment IME. The people shouting loudly about how Twitter must have been so bloated are really just shouting their obvious inexperience working at global scales or their localized ambitions. Could there be too many employees at Twitter? Sure. Most companies have dead weight. The number who were "extra" is probably not 9/10ths the employees though.
- mopsi 4y agoA script that takes text from input field and displays it along with previous submissions in chronological order is trivial; the difficulty comes from serving millions of people.
- motoxpro 4y agoIf twitter, a top 10 viewed site in the world, isn't difficult, what is?
- rzz3 4y agoTo be honest, growth. I work at a tech company of a similar size, and keeping the lights on, so to speak, from a systems perspective is something we can do with a few engineers. By far the bulk of employees at my company are in R&D, sales, business operations, infrastructure growth, etc. If you build a resilient architecture, keeping something like Twitter simply running wouldn’t take that many people.
- itsoktocry 4y ago>If you build a resilient architecture, keeping something like Twitter simply running wouldn’t take that many people. It takes an army of engineers to build a resilient architecture at Twitter's scale. And why are we even talking about "keeping the lights on"? Elon is claiming he's going to build a better video platform than YouTube, complete with better tools and for creators, for crying out loud.
- withinboredom 4y agoHe also said we’d have robo taxi’s by now. I’m not sure how much faith you can put into anything he says because so little of anything he has said has been true. Hell, if he told me I was fired, I probably wouldn’t believe him. Sadly, HR probably would.
- chasd00 4y agoi see your point but you should always put > 0 faith in Musk IMO. No robo taxi's but, even after being laughed out of the room quite literally, landing orbital class boosters is common place now. I don't doubt there's the talent at Twitter required to do amazing things and Musk seems to find ways to push and motivate talent to crazy levels of results. I'm not saying things like a more youtube than youtube is destiny but don't completely write it off.
- Yoric 4y agoNot tech but I'm convinced that just answering LEO requests from all over the world takes at least 500 people. That's without their managers, payrolls, etc. Likewise, bringing in Ad money would be a few more hundreds, because you need to chase leads in all countries. Getting the Ads to work? That's tech and I'd be surprised if it was less than 100 people, too.
- bushbaba 4y agoTracking ads conversions, targeting, refinement of models, latency reduction , data analytics, ads sales, sales ops, etc. yeah easily 1000 employees all in.
- Yoric 4y agoI was only thinking of getting ads to work (from displaying ad previews to announcers to actually delivering them). If you add all the sales, analytics and performance, yeah, that climbs quickly.
- tiahura 4y agoNot tech but I'm convinced that just answering LEO requests from all over the world takes at least 500 people. I subpoena telcos all the time. My sense is that the number is closer to 2 to 3 dozen.
- Yoric 4y agoFor the entire world? How does someone from an office in (say) Morocco checks whether a subpoena equivalent from (for instance) Lybia is legit?
- donkeyd 4y agoFor a second I was confused how Starlink requests were relevant for Twitter. Then I realized you meant 'law enforcement officer' not 'low-earth orbit'.
- Yoric 4y ago
- threeseed 4y ago> I am convinced you can run the entire tech stack with a team of a 100 people Please tell us in detail about the Twitter stack. Because I always find it fascinating how people think they can estimate the effort to maintain it whilst having next to no understanding what so ever of the tech stack.
- WJW 4y agoI assume the reason for statements like that goes something like: 1. A single person can run a mastodon instance in their spare time. Spinning up some containers for the app, a background worker and a database is quite simple. 2. Modern devops tooling makes it fairly trivial to spin up 10k instances of a container instead of 1, by just altering a number in a k8s manifest somewhere. 3. Ergo, a single person equipped with modern tooling (and sufficient funding) could spin up any number of mastodon instances. 4. Twitter is just a big mastodon instance. 5. Now that keeping everything up is sorted, add another 99 devs for feature development and you are done. Now this is obviously faulty logic because points 3 and 4 are very false, but they look reasonable enough at first glance.
- brobdingnagians 4y agoThat is a straw man. There are other perfectly valid interpretations and distributions of a statement that 100 could maintain it, like this: 15 database admins 10 linux sys admins 5 kubernetes specialists 10 windows tech support 25 front end developers 15 back end developers w/ Scala 10 machine learning experts Whether that makeup could or couldn't do it is a different question, or whether it would be a different mix; all of that is up for debate, but the 1/99 ratio is just one very specific, extreme, and laughable mix for anyone who has supported a system of any real size.
- lightbendover 4y agoHow is that valid for creating and maintaining Twitter? 15 backend engineers are not even sufficient for any single facet of Twitter. Plus, there are other initiaves that Twitter takes on that are probably outside the scope of what you consider Twitter. Take a look at just their open source initiatives: https://opensource.twitter.dev/projects/ https://opensource.twitter.dev/projects/. Go to, e.g., Finagle's Github page and look at the number of contributors and commits. While you may disagree that all of this is essential for creating and maintaining Twitter, there are problems that creep in when your platform supports billions of users that are not solved either correctly or at all by existing libraries and services; plus, open source projects serve as important recruiting collateral for engineers, especially for companies that do not have a set of widely used tools that entrench engineers (e.g. AWS, GC, Azure). Creating a husk of an app that looks vaguely like Twitter and supports hundreds of users is a weekend project for anyone with a modicum of talent; building a platform that supports billions of users and is monetized well enough to support itself is an entirely different beast.
- matwood 4y agoFor US only? This sounds like the comment complaining about app size when an Uber engineer showed up and talked about handling 100s of methods of worldwide payments. Many in the US seem to think the internet only exist in the US, and that US rules apply everywhere.
- KoftaBob 4y agoFor comparison, Mastodon is built/maintained by 732 contributors: https://github.com/mastodon/mastodon/graphs/contributors https://github.com/mastodon/mastodon/graphs/contributors
- memish 4y agoGeorge Hotz thinks it's even less, on the order of 20-40 engineers. "20 with cloud, 40 without. So much overlap between iOS, Android, and the web, three people can do all three. More for the backend." https://twitter.com/realGeorgeHotz/status/1593713723675357184 https://twitter.com/realGeorgeHotz/status/159371372367535718...
- stonemetal12 4y agoGoogling "how many people work at stack overflow" gives a number less than 400. If true I think you could put together and run Twitter with less than 400 people from an engineering stand point. However Twitter probably has a much higher soft skills head count for moderation and what not. Though it wouldn't surprise me if a lot of that was contractors not direct head count.
- MockObject 4y agoI don't see any comparison at all. Does Stack have to maintain mobile apps? Their moderation is mostly handled by users, but Twitter must write and maintain tools for handling that. I doubt anyone is trying to upload illegal porn to Stack. Latency of 30 minutes would be noticed by very few Stack users. Twitter probably really needs at least 10x the SREs of Stack.
- stonemetal12 4y agoYeah, but that 400 is everyone at stack not just engineering. A team to put together a mobile app should be less than 10. >Their moderation Above I assumed their moderation team was probably larger than their engineering team, and mostly contractors. Thus I kept my estimate to the size of their engineering team.