5 ms·

Ex-Amazon SDE here. The same happens in FAANGS: tons of software is written and served internally. It's not NIH syndrome, usually. It's about having control ov

Ex-Amazon SDE here. The same happens in FAANGS: tons of software is written and served internally. It's not NIH syndrome, usually. It's about having control over the whole software supply chain for security, reliability, licensing compliance and general quality.

Twirrim 6y ago
As another ex-Amazon... there's a whole bunch of NIH going on there too. The way I saw it, it tended to be split three ways: 1) NIH. Almost always the problems that need solved are interesting, and engineers are naturally chomping at the bit to solve them. Added bonus you can potentially make a name for yourself. This happens way more than it should, in cases that don't meet the other two ways I saw. Solving problems that have already been solved very effectively and efficiently, in a mature low friction fashion. 2) It doesn't scale to needs. A lot of software just doesn't scale to the requirements of the platform. It's hard to understate just how much traffic and work a lot of Amazon infrastructure has to handle. Most software doesn't scale that well because it's not run in so big an environment. We're using some well known commercial software at my current employers (because it works, has a good reputation, and did everything we need), that is experiencing major scaling issues because we're literally orders of magnitude larger than any of their other customers. We're seeing stuff they've never had to deal with before. We're not even close to Amazon's scale for this particular type of software. 3) Need to control the entire software stack, have the ability to drastically modify it to meet the changing demands placed on it. A lot of public software is written to meet one need, and it rarely changes that drastically over time. That's not what the consumers want, even though needs change over time. Change your software too much and you'll lose your existing users that fundamentally need what the software is providing. You can see the boom and fall of it all with so many projects. Take a look at what's happened with Chef and Puppet, for a quick off-the-top-of-my-head example.
- srtjstjsj 6y ago
  4) the most important reason: open source stuff isn't built to integrate with the decades old custom stack cruft
- ex_amazon_sde 6y ago
  > there's a whole bunch of NIH going on there too That's why I wrote "usually".
dragonwriter 6y ago
> It's not NIH syndrome, usually. It's about having control over the whole software supply chain for security, reliability, licensing compliance and general quality. Maybe, though that's usually the exact rationalization given for NIH syndrome. I mean, “NIH syndrome” is never the stated reason for anything.
- notacoward 6y ago
  Bingo. I recently left my job at a FAANG, and there are the top three reasons I saw for code to be written locally. (1) Need to interact with other internal systems. (2) Pure NIH. (3) Need to scale further than outside solutions. 1 and 3 are closely related. There are quite a few legitimate category-3 internal services for provisioning, configuration, service discovery, monitoring, upgrades at various levels, fault remediation, etc. Any other production service would have to interact with most or all of them. It's often easier to build something local than to add all of those "touch points" to an open-source project. I know because I did both while I was there. But pure NIH is very close behind as a reason. Despite all protestations to the contrary, engineers get far more "impact" for creating new things than for fixing old ones. It's hard to get somebody to do X when their bonuses and raises are better served by doing !X. This ends up amplifying, instead of attenuating, the natural impulse of all engineers everywhere to build new things because it's more fun. People always make up other reasons, and perhaps even believe those reasons themselves, but nine times out of ten those reasons are pure delusion.
  - ex_amazon_sde 6y ago
    > Despite all protestations to the contrary, engineers get far more "impact" for creating new things than for fixing old ones. In general, yes, and that's a problem. Luckily in some teams it's much better. > but nine times out of ten those reasons are pure delusion. If you were in a company/team with such level of true NIH syndrome it's good you left.

Twirrim 6y ago

As another ex-Amazon... there's a whole bunch of NIH going on there too. The way I saw it, it tended to be split three ways: 1) NIH. Almost always the problems that need solved are interesting, and engineers are naturally chomping at the bit to solve them. Added bonus you can potentially make a name for yourself. This happens way more than it should, in cases that don't meet the other two ways I saw. Solving problems that have already been solved very effectively and efficiently, in a mature low friction fashion. 2) It doesn't scale to needs. A lot of software just doesn't scale to the requirements of the platform. It's hard to understate just how much traffic and work a lot of Amazon infrastructure has to handle. Most software doesn't scale that well because it's not run in so big an environment. We're using some well known commercial software at my current employers (because it works, has a good reputation, and did everything we need), that is experiencing major scaling issues because we're literally orders of magnitude larger than any of their other customers. We're seeing stuff they've never had to deal with before. We're not even close to Amazon's scale for this particular type of software. 3) Need to control the entire software stack, have the ability to drastically modify it to meet the changing demands placed on it. A lot of public software is written to meet one need, and it rarely changes that drastically over time. That's not what the consumers want, even though needs change over time. Change your software too much and you'll lose your existing users that fundamentally need what the software is providing. You can see the boom and fall of it all with so many projects. Take a look at what's happened with Chef and Puppet, for a quick off-the-top-of-my-head example.

srtjstjsj 6y ago
4) the most important reason: open source stuff isn't built to integrate with the decades old custom stack cruft
ex_amazon_sde 6y ago
> there's a whole bunch of NIH going on there too That's why I wrote "usually".

dragonwriter 6y ago

> It's not NIH syndrome, usually. It's about having control over the whole software supply chain for security, reliability, licensing compliance and general quality. Maybe, though that's usually the exact rationalization given for NIH syndrome. I mean, “NIH syndrome” is never the stated reason for anything.

notacoward 6y ago
Bingo. I recently left my job at a FAANG, and there are the top three reasons I saw for code to be written locally. (1) Need to interact with other internal systems. (2) Pure NIH. (3) Need to scale further than outside solutions. 1 and 3 are closely related. There are quite a few legitimate category-3 internal services for provisioning, configuration, service discovery, monitoring, upgrades at various levels, fault remediation, etc. Any other production service would have to interact with most or all of them. It's often easier to build something local than to add all of those "touch points" to an open-source project. I know because I did both while I was there. But pure NIH is very close behind as a reason. Despite all protestations to the contrary, engineers get far more "impact" for creating new things than for fixing old ones. It's hard to get somebody to do X when their bonuses and raises are better served by doing !X. This ends up amplifying, instead of attenuating, the natural impulse of all engineers everywhere to build new things because it's more fun. People always make up other reasons, and perhaps even believe those reasons themselves, but nine times out of ten those reasons are pure delusion.
- ex_amazon_sde 6y ago
  > Despite all protestations to the contrary, engineers get far more "impact" for creating new things than for fixing old ones. In general, yes, and that's a problem. Luckily in some teams it's much better. > but nine times out of ten those reasons are pure delusion. If you were in a company/team with such level of true NIH syndrome it's good you left.