7 ms·

Less than 20% of Twitter's microservices are needed for it to work

57 points by nstart 4y ago

mikkergp 4y ago
So one theory about Musk’s approach is that you cut down twitter to just it’s core functionality. So what do those other 80% of micro services do? Content moderation, spam, analytics. Presumably some of that infrastructure is related to advertising. But so far all of his numbers seem random and inflated, but I’m guess his strategy is going to be shut a bunch of stuff off and see what you don’t need. Fair enough when they’re in the dire financial straights they are. Is it just me though or is part of this bike shedding on micro services vs monolith. Not really enough information to determine. Edit: I hope all their circuit breakers work correctly.
- _djo_ 4y ago
  Yup. Thing is though, nothing about their financial situation justifies this level of recklessness. Yes, there’s an urgent need to cut costs and find new sources of revenue, thanks to Musk’s own decisions to perform a leveraged buyout and then scare off advertisers. But that doesn’t justify making such sweeping changes in so short a time that there’s no way a proper due diligence is being done. Honestly, I suspect the issue is much simpler. I think Musk detests and can’t bring himself to respect Twitter’s engineers at all, thanks to preconceptions he developed on the outside. That’s why he’s so happy to mock and trash their work in public, why he wanted to get rid of half of them without even giving them the courtesy of speaking to them, and why he believes he knows better. Expect most of those remaining to be replaced as soon as Musk finds other willing saps to take their places. It’s the same reason so many of Musk’s fans, including a bunch of people on HN, reacted with such evident glee at seeing Twitter engineers being laid off. That’s not a healthy mindset.
version_five 4y ago
I hope he turns off the microservices that deliver the "Twitter is better in the app" popup, and block my screen and tell me to sign up if I try scrolling down. As a matter of fact, going web-only would be a great way to streamline
- luuuzeta 4y ago
  I hope he also turns off the one that takes half the screen on mobile and asks me to sign in with a Google account. So annoying.
- bilalq 4y ago
  That's something that's done purely client side. Either modifying in devtools or using ublock origin's zapper tool lets you hide that. There are 3 or 4 elements needed to be erased to make everything work properly after the popup appears.
mytailorisrich 4y ago
I don't know about Twitter specifically, although the fact that Musk thinks they can run the company with half the pre-sale workforce gives an idea, but when I see the size of the engineering teams at many of those companies as reported in the media (I remember for instance that the size of the Uber team is in the thousands, or that Netflix runs 1,000 microservices), I am not surprised that there may be significant bloat.
- luuuzeta 4y ago
  I don't understand why you're being downvoted. Even though it might not the case with all tech companies, there's the possibility there might be significant bloat with some of them.
  - _djo_ 4y ago
    80% bloat though? Given that Twitter had a team whose job it was to analyse the usage of running services to identify savings opportunities? I don’t believe it.
    - sidibe 4y ago
      Well that's because you're starting from a premise that there are competent people that aren't named Elon Musk. From first principles you are already off to a bad start, most of the world is just flailing around waiting for him to come simplify their obvious mistakes.
  - dntrkv 4y ago
    Because "there might be bloat in some companies" is not a very useful, interesting, or enlightening comment. Almost every large company in the world will have some "bloat" Also, "How does Company X have Y engineers? What do they all do?" Is one of the most common and annoying comments that constantly appear in HN threads.
    - mytailorisrich 4y ago
      It's only annoying if you don't try think about it. If you've been in the industry long enough you have experience and you get at least a feel whether x engineers seem a lot or whether y microservices seem a lot. Now in this case we also know that Musk has cut half the workforce so, rightly or wrongly, he has concluded that there is a lot of bloat at Twitter (not 'some company'). I can only take this as a significant piece of evidence as I don't know Twitter. More profoundly I think this has been a consequence of the cult of 'growth'. You have to show that you're growing so you have to grow the team, you have to keep hiring and justify it by 'growth' because if you don't your company will look badly in itself and compared to peers, your IPO will not be so great, your share price will tank. Once you're down that rabbit hole you're stuck. Maybe younger engineers who have only lived professionally through this period actually believe the hype. So I think the current wave of tech lay-offs is partly caused by this no longer being the case so that companies can be more realistic and cut bloat without fear of being seen as a lame duck. Of course a bloated number of engineers means bloated software because all those people were busy doing something, may something not useful at all but something (including perhaps thousands of microservices...).
downrightmike 4y ago
xhkkffbf 4y ago
I've seen this kind of issue first hand. At one project, the platform team spent all of their time concocting ways to create new microservices. It's not a bad idea, but on some days I think it was just designed to sell AWS instances via Kubernetes.
stuckinhell 4y ago
That is just crazy (if true), I always suspected bloat at these big silicon valley companies, but that 20% is just insane.
- stockerta 4y ago
  nah, He just doesn't know what hes talking about.
lm28469 4y ago
20% of a tesla components are needed to go from A to B, yet you still very much like having headlights, a windshield, a roof, blinkers, &c. But as always, 80% of what he tweets he doesn't even believe in and just does it to pump his companies' value
sdfg413t 4y ago
_djo_ 4y ago
That he puts microservices scare quotes says it all, really. He clearly doesn’t understand modern large scale software architectures and is falling back on his archaic, outdated, and limited experience. This isn’t the guy you want making these kinds of decisions. Even if there is bloat in services, which there inevitably is at any large company that’s been around for a while, it’s extremely unlikely that it’s 80% of them. Especially as Twitter’s infra team have in the past already done audits of running services to try to save on operational costs. I wouldn’t be surprised if we see a serious outage or two in the coming couple of weeks.
- BulgarianIdiot 4y ago
  Microservices are simply... services. The term is stupid, but of course you get used to anything if you say it few thousand times. I remember how long I wrote "cloud" in quotes when a decade back or so decided that a remote API from a datacenter should be called a "cloud" for some effin unknowable reason. But I got used to it. Whether Twitter's architecture is good or not is unknowable to us. Microservice overload is an anti-pattern, not something to encourage. But in context it can make sense.
  - _djo_ 4y ago
    Point is a tech CEO in 2022 should not be using scare quotes around microservices. Nor is Twitter’s infrastructure a black box, as there’ve been plenty of talks, blog posts, and other material that explains substantial portions of the fundamental architecture. We know why they adopted GraphQL as the client interface, we know why they’ve used Strato, Thrift, etc. In fact I’d go so far as to say that the average mildly interested developer who’s been reading Twitter’s developer blogs has a much better idea how how Twitter actually works under the hood than its new CEO demanding they get rid of 80% of services. None of this means Twitter’s architecture is above criticism of course. There’s undoubtedly legacy stuff and tech debt there. But it’s unsightly and unfair for Musk to be publicly denigrating the engineers of the company he now owns, purely because he can’t understand how things work.
    - jfray2k22 4y ago
_djo_ 4y ago
dang, can we please change the title? This one takes Musk’s claim at face value, and is not a reliable enough statement of fact to be the headline.
- sn0w_crash 4y ago
  Why is the owner of the company not reliable source? One may not be ideologically aligned with Musk but to pretend that his tweet on the company’s inner workings is somehow less reliable than someone on HN defies logic.
  - cldellow 4y ago
    An owner is well-positioned to know the truth, but also has strong incentives to mislead to serve their own purpose. (See also: all the posts about crypto exchanges recently. I'm not saying Musk is a scammer, more that there are literally examples of why owners aren't reliable sources on the front page right now.)
  - _djo_ 4y ago
    Because: - Musk still has an axe to grind to justify his claim that Twitter was catastrophically mismanaged. - He has a history of making very false statements about Twitter, including most recently that it’s the world’s biggest driver of ad traffic. That harms his credibility. - He has been CEO for what, two weeks? And he’s suddenly discovered all this bloat stuff that others missed? Again that seems unlikely. - This is a claim that on the face of it doesn’t pass muster, given what’s known about Twitter’s engineering approach. That should introduce more scepticism.
    - sn0w_crash 4y ago
      Oookay, that’s going to take me a lot of mental gymnastics. I’m gonna go with Occam’s razor here and just assume the owner of the company knows more about this than I do.
      - deleted 4y ago
        [deleted]
sschueller 4y ago
But how many of those are required to make money?
spcldvlpr 4y ago
Elon is creating Twitter MVP again? Small and agile Startup? Or a Reboot?

57 points by nstart 4y ago

So one theory about Musk’s approach is that you cut down twitter to just it’s core functionality. So what do those other 80% of micro services do? Content moderation, spam, analytics. Presumably some of that infrastructure is related to advertising. But so far all of his numbers seem random and inflated, but I’m guess his strategy is going to be shut a bunch of stuff off and see what you don’t need. Fair enough when they’re in the dire financial straights they are. Is it just me though or is part of this bike shedding on micro services vs monolith. Not really enough information to determine. Edit: I hope all their circuit breakers work correctly.

_djo_ 4y ago
Yup. Thing is though, nothing about their financial situation justifies this level of recklessness. Yes, there’s an urgent need to cut costs and find new sources of revenue, thanks to Musk’s own decisions to perform a leveraged buyout and then scare off advertisers. But that doesn’t justify making such sweeping changes in so short a time that there’s no way a proper due diligence is being done. Honestly, I suspect the issue is much simpler. I think Musk detests and can’t bring himself to respect Twitter’s engineers at all, thanks to preconceptions he developed on the outside. That’s why he’s so happy to mock and trash their work in public, why he wanted to get rid of half of them without even giving them the courtesy of speaking to them, and why he believes he knows better. Expect most of those remaining to be replaced as soon as Musk finds other willing saps to take their places. It’s the same reason so many of Musk’s fans, including a bunch of people on HN, reacted with such evident glee at seeing Twitter engineers being laid off. That’s not a healthy mindset.

version_five 4y ago

I hope he turns off the microservices that deliver the "Twitter is better in the app" popup, and block my screen and tell me to sign up if I try scrolling down. As a matter of fact, going web-only would be a great way to streamline

luuuzeta 4y ago
I hope he also turns off the one that takes half the screen on mobile and asks me to sign in with a Google account. So annoying.
bilalq 4y ago
That's something that's done purely client side. Either modifying in devtools or using ublock origin's zapper tool lets you hide that. There are 3 or 4 elements needed to be erased to make everything work properly after the popup appears.

mytailorisrich 4y ago

I don't know about Twitter specifically, although the fact that Musk thinks they can run the company with half the pre-sale workforce gives an idea, but when I see the size of the engineering teams at many of those companies as reported in the media (I remember for instance that the size of the Uber team is in the thousands, or that Netflix runs 1,000 microservices), I am not surprised that there may be significant bloat.

luuuzeta 4y ago
I don't understand why you're being downvoted. Even though it might not the case with all tech companies, there's the possibility there might be significant bloat with some of them.
- _djo_ 4y ago
  80% bloat though? Given that Twitter had a team whose job it was to analyse the usage of running services to identify savings opportunities? I don’t believe it.
  - sidibe 4y ago
    Well that's because you're starting from a premise that there are competent people that aren't named Elon Musk. From first principles you are already off to a bad start, most of the world is just flailing around waiting for him to come simplify their obvious mistakes.
- dntrkv 4y ago
  Because "there might be bloat in some companies" is not a very useful, interesting, or enlightening comment. Almost every large company in the world will have some "bloat" Also, "How does Company X have Y engineers? What do they all do?" Is one of the most common and annoying comments that constantly appear in HN threads.
  - mytailorisrich 4y ago
    It's only annoying if you don't try think about it. If you've been in the industry long enough you have experience and you get at least a feel whether x engineers seem a lot or whether y microservices seem a lot. Now in this case we also know that Musk has cut half the workforce so, rightly or wrongly, he has concluded that there is a lot of bloat at Twitter (not 'some company'). I can only take this as a significant piece of evidence as I don't know Twitter. More profoundly I think this has been a consequence of the cult of 'growth'. You have to show that you're growing so you have to grow the team, you have to keep hiring and justify it by 'growth' because if you don't your company will look badly in itself and compared to peers, your IPO will not be so great, your share price will tank. Once you're down that rabbit hole you're stuck. Maybe younger engineers who have only lived professionally through this period actually believe the hype. So I think the current wave of tech lay-offs is partly caused by this no longer being the case so that companies can be more realistic and cut bloat without fear of being seen as a lame duck. Of course a bloated number of engineers means bloated software because all those people were busy doing something, may something not useful at all but something (including perhaps thousands of microservices...).

downrightmike 4y ago

xhkkffbf 4y ago

I've seen this kind of issue first hand. At one project, the platform team spent all of their time concocting ways to create new microservices. It's not a bad idea, but on some days I think it was just designed to sell AWS instances via Kubernetes.

stuckinhell 4y ago

That is just crazy (if true), I always suspected bloat at these big silicon valley companies, but that 20% is just insane.

stockerta 4y ago
nah, He just doesn't know what hes talking about.

lm28469 4y ago

20% of a tesla components are needed to go from A to B, yet you still very much like having headlights, a windshield, a roof, blinkers, &c. But as always, 80% of what he tweets he doesn't even believe in and just does it to pump his companies' value

sdfg413t 4y ago

_djo_ 4y ago

That he puts microservices scare quotes says it all, really. He clearly doesn’t understand modern large scale software architectures and is falling back on his archaic, outdated, and limited experience. This isn’t the guy you want making these kinds of decisions. Even if there is bloat in services, which there inevitably is at any large company that’s been around for a while, it’s extremely unlikely that it’s 80% of them. Especially as Twitter’s infra team have in the past already done audits of running services to try to save on operational costs. I wouldn’t be surprised if we see a serious outage or two in the coming couple of weeks.

BulgarianIdiot 4y ago
Microservices are simply... services. The term is stupid, but of course you get used to anything if you say it few thousand times. I remember how long I wrote "cloud" in quotes when a decade back or so decided that a remote API from a datacenter should be called a "cloud" for some effin unknowable reason. But I got used to it. Whether Twitter's architecture is good or not is unknowable to us. Microservice overload is an anti-pattern, not something to encourage. But in context it can make sense.
- _djo_ 4y ago
  Point is a tech CEO in 2022 should not be using scare quotes around microservices. Nor is Twitter’s infrastructure a black box, as there’ve been plenty of talks, blog posts, and other material that explains substantial portions of the fundamental architecture. We know why they adopted GraphQL as the client interface, we know why they’ve used Strato, Thrift, etc. In fact I’d go so far as to say that the average mildly interested developer who’s been reading Twitter’s developer blogs has a much better idea how how Twitter actually works under the hood than its new CEO demanding they get rid of 80% of services. None of this means Twitter’s architecture is above criticism of course. There’s undoubtedly legacy stuff and tech debt there. But it’s unsightly and unfair for Musk to be publicly denigrating the engineers of the company he now owns, purely because he can’t understand how things work.
  - jfray2k22 4y ago

_djo_ 4y ago

dang, can we please change the title? This one takes Musk’s claim at face value, and is not a reliable enough statement of fact to be the headline.

sn0w_crash 4y ago
Why is the owner of the company not reliable source? One may not be ideologically aligned with Musk but to pretend that his tweet on the company’s inner workings is somehow less reliable than someone on HN defies logic.
- cldellow 4y ago
  An owner is well-positioned to know the truth, but also has strong incentives to mislead to serve their own purpose. (See also: all the posts about crypto exchanges recently. I'm not saying Musk is a scammer, more that there are literally examples of why owners aren't reliable sources on the front page right now.)
- _djo_ 4y ago
  Because: - Musk still has an axe to grind to justify his claim that Twitter was catastrophically mismanaged. - He has a history of making very false statements about Twitter, including most recently that it’s the world’s biggest driver of ad traffic. That harms his credibility. - He has been CEO for what, two weeks? And he’s suddenly discovered all this bloat stuff that others missed? Again that seems unlikely. - This is a claim that on the face of it doesn’t pass muster, given what’s known about Twitter’s engineering approach. That should introduce more scepticism.
  - sn0w_crash 4y ago
    Oookay, that’s going to take me a lot of mental gymnastics. I’m gonna go with Occam’s razor here and just assume the owner of the company knows more about this than I do.
    - deleted 4y ago
      [deleted]

sschueller 4y ago

But how many of those are required to make money?

spcldvlpr 4y ago

Elon is creating Twitter MVP again? Small and agile Startup? Or a Reboot?