4 ms·

I'd really like to see someone smart trying to estimate what 550 engineers are working at Twitter on. It still seems like plenty.

ramraj07 4y ago
This trite argument made some sense when there were 8000 engineers, but when the 4th most popular social media platform has only 550 engineers working on it, you don’t need to dig deep to figure out how so many engineers are needed. Even the famously nimble WhatsApp which has fewer features and complexity needs this order of magnitude engineers so why would you expect Twitter to be leaner?
- marcos100 4y ago
  Didn't WhatsApp had like 50 employees in total when they were sold to Facebook?
- zoover2020 4y ago
  WhatsApp having less complexity? Care to elaborate on that? I'm aware they scaled to 1B+ users with ~100 engineers but I'm failing to see why they'd be working on less complex problems.
  - Glaurung 4y ago
    Twitter has a direct messaging feature. It also has many, many other features. Whatsapp is direct messaging, full stop. The two are not at all comparable.
  - no_way 4y ago
    WhatsApp always comes up in comparison to Twitter, but they are fundamentally different. WhatsApp is mostly one on one interaction, while Twitter potentially allows every user to interact with every other user, it's unbelievably more complex to do that at scale compared to WhatsApp, not even talking about all the extra features Twitter has. Search probably being best example of complexity differences.
    - bsaul 4y ago
      what's app has e2e encryption, on all platforms including web. Which requires strong constraints on delivery ordering. Operating on almost all native platforms, at this scale, with those constraints, isn't something i find much easier than building a feed.
      - flashgordon 4y ago
        Out of curiosity what makes e2e encryption impact ordering guarantees? With feed things like ranking (bunch of ml pipelines) etc make it pretty stochastic? On Whatsapp the ordering guarantees are limited to within the thread and server side timestamps are good enough for the most part to enforce it. Whatsapp/messaging is a very horizontally scalable problem. With feed (assuming) you have a fully personal feed the storage alone is a pain to manage and optimize.
        bsaul 4y ago
        i'd say key exchange. You need to get key information and message in the correct order orelse you won't be able to decode the info. Now of course you can reimplement an ordering mechanism on top of an unordered one, with lost messages detection etc on the client side, and let the server be loose. but that means reimplementing that logic on every client codebase.
- comprev 4y ago
  WhatsApp had a surprisingly small number of engineers upon acquisition - around 50 or so?
  - Glaurung 4y ago
    I imagine the team responsible for maintaining the direct messaging feature of twitter is also around 50 or so people. In case you haven't noticed, twitter has many, many, many other features beyond direct messaging, each of which requires its own team to maintain, and many of which are far more complex than a messaging client.
  - mytailorisrich 4y ago
    Yes, 55 employees iirc. I don't think it was 'surprisingly small', though. It's just that cash-rich startups have had the habit of hiring as much as they could instead of hiring the minimum required, which is actually the sensible way to manage a company.
  - toto 4y ago
    Ambiance was different compared to today: https://twitter.com/TikTokInvestors/status/1615189880321241088 https://twitter.com/TikTokInvestors/status/16151898803212410... Whatsapp had 35 engineers for 450M users at the time of its acquisition by FB (2014). It had 50 engineers for 900M users (2015). https://www.wired.com/2015/09/whatsapp-serves-900-million-users-50-engineers/#:~:text=When%20the%20company%20was%20acquired,almost%20entirely%20on%20its%20own https://www.wired.com/2015/09/whatsapp-serves-900-million-us...
  - p_l 4y ago
    WhatsApp also had no need for moderation or ads handling (IIRC, they used small yearly payments plus contracts with network operators). Just handling the ads, both backend and user interfaces for buyers, would quickly net you several more developers. It's easy to have only 45 engineers when you constrain the scope of what they have to do very well, but the scope was much, much, much smaller than Twitter's.
- michpoch 4y ago
  Still; numbers. I do not use Twitter, but the web app is relatively simple; I am not hearing much good about the mobile apps. So we got what; maybe 30 people working on the frontends? I do not think Twitter had ever 8000 engineers - my understanding is that there were that many employees. It's easy to imagine a lot of non-tech roles, b2b sales, management, support etc.
- yucky 4y ago
  This isn't logical though. We have evidence that 8,000 was far too many, because they're gone and Twitter is still Twitter. We have zero evidence that 550 isn't still too many.
  - p_l 4y ago
    Dunno, ever since Musk started purges and especially after infamous "I'll switch off a bunch of servers" thing, I observe more and more bugs on Twitter. Mind you, I already had searches that triggered bugs in mobile app where first search would always fail, but things like significant drop in tweets I see in my feed from people I follow (and no, none of them were bots), random outages like Lists or the whole search feature just failing, pervasive feeling that even searches return maybe 1/10th of what I was getting before... This is a tirefire.
vkou 4y ago
You don't need 550 engineers to build Twitter, the 140 character-message app. Unfortunately, there's a lot more than building a messaging app to running Twitter, the business that sells ads next to those messages.
mrtksn 4y ago
Run a database and a UI to put and show things from it: 0 engineers to run after installation. A full stack hacker can build it. Run the same thing but for number of people larger than a single server can handle: A an engineer or two for bug catching and resource allocations and a few engineers to build it using software tools for caching and load balancing. The same thing but tens of millions around the globe: A few engineers to run the thing, A few to keep an eye on the network around the globe and access quality, a few to mitigate hostile behaviour. To build the thing you need a few people with knowledge on distributed systems and specialised people on 3rd party products for CDN and its management. Of course you need the people to build the thing and its UI too. Oh, so you want to make money from it? You need people to build tools which will interface with your thing to deliver things like ads and special functions. Now you need to replicate all of the previous stuff but this time for your money making software. Europe says data collection should be like X, UK says should be like Y and the USA should be like Z. Now the people who built the thing can't go home and leave it to the maintainers because they need to modify how things work all the time. Israel wants to know some info about someone, Turkey wants the some specific data collected and some tweets must not be visible in Turkey, Saudis say they don't want some type of tweets to exist etc. Now you need people to build these things when still don't mess up with the EU, UK, USA practices. Once again you replicate a lot of people but with special duties for special requirements. Oh crap, some pedos are posting very illegal stuff and others are running massive spam or political opinion manufacturing operations and everyone blames you. You need to create tools to interface with your product to tackle these things. Once again a lot of engineers are needed because you don't simply run an SQL query from your local database, so you need people who understand how to make the thing work on your scale. Something can no longer scale and you need an alternative solution, you build that alternative solution with a team of engineers but the old stuff can't just go away because some other systems depend on it and it will take years to change their thing so maybe you simply keep the old stuff around together with their engineers? Before you know it, hundreds or even thousands of people are working on it. Elon firing content safety team is probably about untangling the mess and not about love for sexual predators or fascist or something. I don't know about the reality of Twitter but having seen how sausage is made in some large companies, I will speculate that it's a build up over time as the nature of the business evolved. Knowing how things work today it's probably possible to simplify it with a complete re-write but how many re-writes go well?

ramraj07 4y ago

This trite argument made some sense when there were 8000 engineers, but when the 4th most popular social media platform has only 550 engineers working on it, you don’t need to dig deep to figure out how so many engineers are needed. Even the famously nimble WhatsApp which has fewer features and complexity needs this order of magnitude engineers so why would you expect Twitter to be leaner?

marcos100 4y ago
Didn't WhatsApp had like 50 employees in total when they were sold to Facebook?
zoover2020 4y ago
WhatsApp having less complexity? Care to elaborate on that? I'm aware they scaled to 1B+ users with ~100 engineers but I'm failing to see why they'd be working on less complex problems.
- Glaurung 4y ago
  Twitter has a direct messaging feature. It also has many, many other features. Whatsapp is direct messaging, full stop. The two are not at all comparable.
- no_way 4y ago
  WhatsApp always comes up in comparison to Twitter, but they are fundamentally different. WhatsApp is mostly one on one interaction, while Twitter potentially allows every user to interact with every other user, it's unbelievably more complex to do that at scale compared to WhatsApp, not even talking about all the extra features Twitter has. Search probably being best example of complexity differences.
  - bsaul 4y ago
    what's app has e2e encryption, on all platforms including web. Which requires strong constraints on delivery ordering. Operating on almost all native platforms, at this scale, with those constraints, isn't something i find much easier than building a feed.
    - flashgordon 4y ago
      Out of curiosity what makes e2e encryption impact ordering guarantees? With feed things like ranking (bunch of ml pipelines) etc make it pretty stochastic? On Whatsapp the ordering guarantees are limited to within the thread and server side timestamps are good enough for the most part to enforce it. Whatsapp/messaging is a very horizontally scalable problem. With feed (assuming) you have a fully personal feed the storage alone is a pain to manage and optimize.
      - bsaul 4y ago
        i'd say key exchange. You need to get key information and message in the correct order orelse you won't be able to decode the info. Now of course you can reimplement an ordering mechanism on top of an unordered one, with lost messages detection etc on the client side, and let the server be loose. but that means reimplementing that logic on every client codebase.
comprev 4y ago
WhatsApp had a surprisingly small number of engineers upon acquisition - around 50 or so?
- Glaurung 4y ago
  I imagine the team responsible for maintaining the direct messaging feature of twitter is also around 50 or so people. In case you haven't noticed, twitter has many, many, many other features beyond direct messaging, each of which requires its own team to maintain, and many of which are far more complex than a messaging client.
- mytailorisrich 4y ago
  Yes, 55 employees iirc. I don't think it was 'surprisingly small', though. It's just that cash-rich startups have had the habit of hiring as much as they could instead of hiring the minimum required, which is actually the sensible way to manage a company.
- toto 4y ago
  Ambiance was different compared to today: https://twitter.com/TikTokInvestors/status/1615189880321241088 https://twitter.com/TikTokInvestors/status/16151898803212410... Whatsapp had 35 engineers for 450M users at the time of its acquisition by FB (2014). It had 50 engineers for 900M users (2015). https://www.wired.com/2015/09/whatsapp-serves-900-million-users-50-engineers/#:~:text=When%20the%20company%20was%20acquired,almost%20entirely%20on%20its%20own https://www.wired.com/2015/09/whatsapp-serves-900-million-us...
- p_l 4y ago
  WhatsApp also had no need for moderation or ads handling (IIRC, they used small yearly payments plus contracts with network operators). Just handling the ads, both backend and user interfaces for buyers, would quickly net you several more developers. It's easy to have only 45 engineers when you constrain the scope of what they have to do very well, but the scope was much, much, much smaller than Twitter's.
michpoch 4y ago
Still; numbers. I do not use Twitter, but the web app is relatively simple; I am not hearing much good about the mobile apps. So we got what; maybe 30 people working on the frontends? I do not think Twitter had ever 8000 engineers - my understanding is that there were that many employees. It's easy to imagine a lot of non-tech roles, b2b sales, management, support etc.
yucky 4y ago
This isn't logical though. We have evidence that 8,000 was far too many, because they're gone and Twitter is still Twitter. We have zero evidence that 550 isn't still too many.
- p_l 4y ago
  Dunno, ever since Musk started purges and especially after infamous "I'll switch off a bunch of servers" thing, I observe more and more bugs on Twitter. Mind you, I already had searches that triggered bugs in mobile app where first search would always fail, but things like significant drop in tweets I see in my feed from people I follow (and no, none of them were bots), random outages like Lists or the whole search feature just failing, pervasive feeling that even searches return maybe 1/10th of what I was getting before... This is a tirefire.

vkou 4y ago

You don't need 550 engineers to build Twitter, the 140 character-message app. Unfortunately, there's a lot more than building a messaging app to running Twitter, the business that sells ads next to those messages.

mrtksn 4y ago

Run a database and a UI to put and show things from it: 0 engineers to run after installation. A full stack hacker can build it. Run the same thing but for number of people larger than a single server can handle: A an engineer or two for bug catching and resource allocations and a few engineers to build it using software tools for caching and load balancing. The same thing but tens of millions around the globe: A few engineers to run the thing, A few to keep an eye on the network around the globe and access quality, a few to mitigate hostile behaviour. To build the thing you need a few people with knowledge on distributed systems and specialised people on 3rd party products for CDN and its management. Of course you need the people to build the thing and its UI too. Oh, so you want to make money from it? You need people to build tools which will interface with your thing to deliver things like ads and special functions. Now you need to replicate all of the previous stuff but this time for your money making software. Europe says data collection should be like X, UK says should be like Y and the USA should be like Z. Now the people who built the thing can't go home and leave it to the maintainers because they need to modify how things work all the time. Israel wants to know some info about someone, Turkey wants the some specific data collected and some tweets must not be visible in Turkey, Saudis say they don't want some type of tweets to exist etc. Now you need people to build these things when still don't mess up with the EU, UK, USA practices. Once again you replicate a lot of people but with special duties for special requirements. Oh crap, some pedos are posting very illegal stuff and others are running massive spam or political opinion manufacturing operations and everyone blames you. You need to create tools to interface with your product to tackle these things. Once again a lot of engineers are needed because you don't simply run an SQL query from your local database, so you need people who understand how to make the thing work on your scale. Something can no longer scale and you need an alternative solution, you build that alternative solution with a team of engineers but the old stuff can't just go away because some other systems depend on it and it will take years to change their thing so maybe you simply keep the old stuff around together with their engineers? Before you know it, hundreds or even thousands of people are working on it. Elon firing content safety team is probably about untangling the mess and not about love for sexual predators or fascist or something. I don't know about the reality of Twitter but having seen how sausage is made in some large companies, I will speculate that it's a build up over time as the nature of the business evolved. Knowing how things work today it's probably possible to simplify it with a complete re-write but how many re-writes go well?