4 ms·

I tend to think this is overblown, if for no other reason than, whatever the team's current status, there are always mercenary, high-rate consultants that can b

I tend to think this is overblown, if for no other reason than, whatever the team's current status, there are always mercenary, high-rate consultants that can be brought in to help.

jstx1 4y ago
Maybe it's overblown, I don't know if consultants are really the fix though. What kind of consultant jumps into a completely new codebase of a massive distributed system with hundreds of millions of users that has broken down to the point of being unusable and manages to fix anything, and how long does that take? Sometimes it takes months for regular new employees to do anything productive in some of these codebases.
- groffee 4y ago
  > Sometimes it takes months for regular new employees to do anything productive in some of these codebases. Which is most probably why most of them were just kicked out. Twitter isn't complicated, it's literally just a micro-blogging platform.
  - jstx1 4y ago
    I think you're underestimating how difficult operating at that scale is. There's no "just" here. It's hundreds of millions of people, all over the world, using your app 24/7, with the expectation that your platform is always there, that it shows consistent data and that it responds instantly. That's very difficult to build and maintain.
  - deleted 4y ago
    [deleted]
  - etchalon 4y ago
    "It's just a banana Michael, how much could it cost?"
- Zigurd 4y ago
  I get paid to jump into completely new (though mostly old) code bases that have problems. But those are mobile apps implemented on a tech stack I'm familiar with. Twitter-scale is totally different. I have a family member who works at keeping major parts of a major platform running. At that scale you make your own tools. Nobody outside the company is ready to use those tools without training.
  - everforward 4y ago
    This. If there's no one left to pass on the tribal knowledge, the contractors have to reverse engineer how that stuff works. It'll take months to rebuild that tribal knowledge. The underlying tech is probably standard (e.g. they probably haven't invented their own routing protocol), but the control plane is likely custom. The impact of making manual changes or changing the control plane is probably unknown, ranging from being fine to borking the system in some way that even reverting the changes won't fix. The best option would be to try to rehire the former employees, probably at dramatically increased pay. I don't even know if that would work, though. Bridges have been burned.
- etchalon 4y ago
  I have no doubt the consultants would be less efficient (perhaps by orders of magnitude) than employees who'd been there for years, but I think they could at least keep the lights on.

I tend to think this is overblown, if for no other reason than, whatever the team's current status, there are always mercenary, high-rate consultants that can be brought in to help.

jstx1 4y ago

Maybe it's overblown, I don't know if consultants are really the fix though. What kind of consultant jumps into a completely new codebase of a massive distributed system with hundreds of millions of users that has broken down to the point of being unusable and manages to fix anything, and how long does that take? Sometimes it takes months for regular new employees to do anything productive in some of these codebases.

groffee 4y ago
> Sometimes it takes months for regular new employees to do anything productive in some of these codebases. Which is most probably why most of them were just kicked out. Twitter isn't complicated, it's literally just a micro-blogging platform.
- jstx1 4y ago
  I think you're underestimating how difficult operating at that scale is. There's no "just" here. It's hundreds of millions of people, all over the world, using your app 24/7, with the expectation that your platform is always there, that it shows consistent data and that it responds instantly. That's very difficult to build and maintain.
- deleted 4y ago
  [deleted]
- etchalon 4y ago
  "It's just a banana Michael, how much could it cost?"
Zigurd 4y ago
I get paid to jump into completely new (though mostly old) code bases that have problems. But those are mobile apps implemented on a tech stack I'm familiar with. Twitter-scale is totally different. I have a family member who works at keeping major parts of a major platform running. At that scale you make your own tools. Nobody outside the company is ready to use those tools without training.
- everforward 4y ago
  This. If there's no one left to pass on the tribal knowledge, the contractors have to reverse engineer how that stuff works. It'll take months to rebuild that tribal knowledge. The underlying tech is probably standard (e.g. they probably haven't invented their own routing protocol), but the control plane is likely custom. The impact of making manual changes or changing the control plane is probably unknown, ranging from being fine to borking the system in some way that even reverting the changes won't fix. The best option would be to try to rehire the former employees, probably at dramatically increased pay. I don't even know if that would work, though. Bridges have been burned.
etchalon 4y ago
I have no doubt the consultants would be less efficient (perhaps by orders of magnitude) than employees who'd been there for years, but I think they could at least keep the lights on.