5 ms·

If you work for a big engineering organization you will find yourself questioning things that are being built a lot. Open Source alternatives often have higher

by msoad 6y ago

If you work for a big engineering organization you will find yourself questioning things that are being built a lot. Open Source alternatives often have higher quality and better support, yet engineering organizations opt to build their own. You might wonder why? It's because growth is the name of the game. Any engineering leader wants bigger and bigger organization under her/him so what they do is they green light projects that mostly don't make sense but they can pretty much lie about how essential it is to build in house to the board/CEO. When winter comes, those projects don't make sense anymore because cost cutting measures are in mandate. The same leader might even make the case for the Open Source alternative. I've seen this enough times to know it is a pattern in our industry.

mrfusion 6y ago
Great point. Very simple explanation.
vikiomega9 6y ago
From your experiences, tell me more! How could this problem be avoided? How does one sniff out such glut?
- neutronicus 6y ago
  If someone comes to you and says "we need to hire more people to do my job so I can move into a leadership role" say "sure, if you take a pay cut," or "no" If you are making a lot of money and one of your employees has performed well, and never asked you to hire more people to do their job, give them a God damn raise
- golergka 6y ago
  Personally, I think this problem is inevitable as organization scales. So, if you don't want to fall into that, remedy is simple: don't scale. As in, you can scale in terms of revenue and user amount, but avoid reinvesting it into hiring more and more people. (Classic examples of companies that achieved that, at least to some extent and at a certain moment, are Instagram and Whatsapp, but there are many more less famous ones). I guess, when you're on the rocket part of the hockey stick it seems that you can solve anything by hiring more people. A lot of leaders kind of forget what they sacrifice when they do that.
- JoshTriplett 6y ago
  > From your experiences, tell me more! How could this problem be avoided? How does one sniff out such glut? To a certain extent: Create a corporate culture where people can get work done without needing to be "protected" from the rest of the company around them Select against empire-building folks. Push a culture of reusing and improving existing tools whenever possible. Recognize and reward people for avoiding and eliminating technical debt, not just for building new things. What you recognize and laud, you will get more of. (There's a balance here: you need both kinds of people.) Hire more collaborators and less "ninjas". If you have enough budget, empower people to experiment with interesting things without having to hide those folks under a corporate-level justification smokescreen. Understand that some explorations and experimentation will not pay off, and support that anyway, so people don't feel like they have to hide things until they're successful. Encourage people to report organizational issues and organizational friction, and fix it as early as possible, so that people don't have to create "shadow infrastructure" to get their job done.
29athrowaway 6y ago
Sometimes the motivation is security. https://en.wikipedia.org/wiki/Trusted_computing_base https://en.wikipedia.org/wiki/Trusted_computing_base In-house solutions are not necessarily more secure, just more obscure/less discoverable.
soumyadeb 6y ago
Great point!! A perfect example is AirBnB - look at the number of open-source projects that came from there. https://airbnb.io/projects/ https://airbnb.io/projects/ I myself have used two of these projects heavily (Airflow, SuperSet) and am thankful to the teams for building such great products. But did this make any business sense for AirBnB to spend on these? In this post COVID world, very unlikely this would continue.
- andrepd 6y ago
  What do you mean post COVID world?
  - softwarejosh 6y ago
    Also, OP is acting like covid trashing the stock market is really affecting all companies equally, lots of tech companies are getting crazy rise
cookiecaper 6y ago
Yes, I've seen this numerous times as well. I call it "vanity hiring". "My department needs 12 new headcount because the other department is getting 10". The sad thing is that you often have to play the game defensively -- I've seen semi-decent organizations end up consumed by the 80% overweight departments because bigger departments end up with all the advantages just as a function of their size, even if they're almost completely worthless. Have to keep pace with the useless hiring just so they can't say "well it's our smallest group, only 5 guys", etc.
- pengaru 6y ago
  > I call it "vanity hiring". It already has a name; empire building
  - cookiecaper 6y ago
    I see "empire building" as a larger campaign. In some cases, vanity hiring is really just vanity hiring, motivated by noting more than the impulse not to let your peer's headcount pull significantly ahead of your own.
  - xnyan 6y ago
    I think you could argue it already has a more general name; individual self interest. The peak headcount you’ve managed or directred has a pretty direct impact on the level of manigeiral roles you are in contention for.
em-bee 6y ago
but it's not always malicious. sometimes building something new seems obvious. i have to push hard to get people to back up that obviousness with evidence. (provide me with evidence that these existing solutions are not suitable before we build a new one from scratch. sometimes building something new seems easier or cheaper than the effort required to evaluate alternatives, especially if the solution only takes a few days or weeks of work (take the obviousness factor above into account). another factor is the seemingly common dislike of working with other peoples code. especially when it's known that modifications will need to be made. do we take this system with a lot of legacy code where we have to modify 20% or is it easier to start over and build what we need?
- keanebean86 6y ago
  There's also the angle that home grown projects can interact better with each other and possibly have a unified look and feel. You could bolt both of those on top of OSS but that can end badly.
- flukus 6y ago
  In house tools can also offer a lot more stability than OSS/commercial equivalents that will often require you to be on their upgrade treadmill. Very few companies will plan for this kind of maintenance so it's important the developers align with that and don't take on unnecessary burdens. There's more OSS than ever but there is a huge lack of stability in most of it.
  - 2ion 6y ago
    This is key. Extremely many "OSS" projects are also not up to quality standards and require significant engineering time for understanding, auditing and perhaps integrating the thing, even before deploying it. The bigger the set of useless features on a project, the bigger the risk in introducing an unknown into the critical paths that allow you to ship/provide a product/service. While many global "base" technologies like Postgres, enterprise Linux, Redis, NodeJS, Tomecat, the JDKs, etc are solid lighthouse projects with better quality than what "just writing your own" without world-class engineering teams and investment could produce in a reasonable amount of time, many others are just not that good. The cost of "adopting" a 2nd- or 3rd-rate FOSS solution may be very high and introduce significant technical debt down the road. In way to many companies, engineers (including engineering managers) have to explain to a non-technical C suite that OSS/FOSS IS NOT FREE and NOT GUARANTEED TO BE CHEAPER than going with proprietary or self-written, minimal, tested and concise solutions in any given context. They can be, of course, and with some engineering investment they can be for the longest time in the feature, but every case is different and a "wave" certainly is not what just removes the need to do this kind of evaluation.
stilisstuk 6y ago
Niskanen and budget maximizing https://en.m.wikipedia.org/wiki/Budget-maximizing_model https://en.m.wikipedia.org/wiki/Budget-maximizing_model Old School political science I have more faith in bureau shaping, which is almost the same.https://en.m.wikipedia.org/wiki/Bureau-shaping_mode https://en.m.wikipedia.org/wiki/Bureau-shaping_mode My it dep is mostly run by an Oss linux geek. This is super cool
dv_dt 6y ago
I think this is true in some cases, but there are others where an engineering org is proposing building an internal solution on an open source stack vs fewer dedicated internal resources with a proprietary solution and the argument becomes one of using internal resources to do something which is maybe a little outside of "core competency" areas vs a proprietary solution that may claim to do everything needed with a risk of limiting options either technically or financially in the future. There I've seen a pattern of short term financial cost overriding future risk, but maybe that shifts a little due to pandemic outcomes.

by msoad 6y ago

mrfusion 6y ago

Great point. Very simple explanation.

vikiomega9 6y ago

From your experiences, tell me more! How could this problem be avoided? How does one sniff out such glut?

neutronicus 6y ago
If someone comes to you and says "we need to hire more people to do my job so I can move into a leadership role" say "sure, if you take a pay cut," or "no" If you are making a lot of money and one of your employees has performed well, and never asked you to hire more people to do their job, give them a God damn raise
golergka 6y ago
Personally, I think this problem is inevitable as organization scales. So, if you don't want to fall into that, remedy is simple: don't scale. As in, you can scale in terms of revenue and user amount, but avoid reinvesting it into hiring more and more people. (Classic examples of companies that achieved that, at least to some extent and at a certain moment, are Instagram and Whatsapp, but there are many more less famous ones). I guess, when you're on the rocket part of the hockey stick it seems that you can solve anything by hiring more people. A lot of leaders kind of forget what they sacrifice when they do that.
JoshTriplett 6y ago
> From your experiences, tell me more! How could this problem be avoided? How does one sniff out such glut? To a certain extent: Create a corporate culture where people can get work done without needing to be "protected" from the rest of the company around them Select against empire-building folks. Push a culture of reusing and improving existing tools whenever possible. Recognize and reward people for avoiding and eliminating technical debt, not just for building new things. What you recognize and laud, you will get more of. (There's a balance here: you need both kinds of people.) Hire more collaborators and less "ninjas". If you have enough budget, empower people to experiment with interesting things without having to hide those folks under a corporate-level justification smokescreen. Understand that some explorations and experimentation will not pay off, and support that anyway, so people don't feel like they have to hide things until they're successful. Encourage people to report organizational issues and organizational friction, and fix it as early as possible, so that people don't have to create "shadow infrastructure" to get their job done.

29athrowaway 6y ago

Sometimes the motivation is security. https://en.wikipedia.org/wiki/Trusted_computing_base https://en.wikipedia.org/wiki/Trusted_computing_base In-house solutions are not necessarily more secure, just more obscure/less discoverable.

soumyadeb 6y ago

Great point!! A perfect example is AirBnB - look at the number of open-source projects that came from there. https://airbnb.io/projects/ https://airbnb.io/projects/ I myself have used two of these projects heavily (Airflow, SuperSet) and am thankful to the teams for building such great products. But did this make any business sense for AirBnB to spend on these? In this post COVID world, very unlikely this would continue.

andrepd 6y ago
What do you mean post COVID world?
- softwarejosh 6y ago
  Also, OP is acting like covid trashing the stock market is really affecting all companies equally, lots of tech companies are getting crazy rise

cookiecaper 6y ago

Yes, I've seen this numerous times as well. I call it "vanity hiring". "My department needs 12 new headcount because the other department is getting 10". The sad thing is that you often have to play the game defensively -- I've seen semi-decent organizations end up consumed by the 80% overweight departments because bigger departments end up with all the advantages just as a function of their size, even if they're almost completely worthless. Have to keep pace with the useless hiring just so they can't say "well it's our smallest group, only 5 guys", etc.

pengaru 6y ago
> I call it "vanity hiring". It already has a name; empire building
- cookiecaper 6y ago
  I see "empire building" as a larger campaign. In some cases, vanity hiring is really just vanity hiring, motivated by noting more than the impulse not to let your peer's headcount pull significantly ahead of your own.
- xnyan 6y ago
  I think you could argue it already has a more general name; individual self interest. The peak headcount you’ve managed or directred has a pretty direct impact on the level of manigeiral roles you are in contention for.

em-bee 6y ago

but it's not always malicious. sometimes building something new seems obvious. i have to push hard to get people to back up that obviousness with evidence. (provide me with evidence that these existing solutions are not suitable before we build a new one from scratch. sometimes building something new seems easier or cheaper than the effort required to evaluate alternatives, especially if the solution only takes a few days or weeks of work (take the obviousness factor above into account). another factor is the seemingly common dislike of working with other peoples code. especially when it's known that modifications will need to be made. do we take this system with a lot of legacy code where we have to modify 20% or is it easier to start over and build what we need?

keanebean86 6y ago
There's also the angle that home grown projects can interact better with each other and possibly have a unified look and feel. You could bolt both of those on top of OSS but that can end badly.
flukus 6y ago
In house tools can also offer a lot more stability than OSS/commercial equivalents that will often require you to be on their upgrade treadmill. Very few companies will plan for this kind of maintenance so it's important the developers align with that and don't take on unnecessary burdens. There's more OSS than ever but there is a huge lack of stability in most of it.
- 2ion 6y ago
  This is key. Extremely many "OSS" projects are also not up to quality standards and require significant engineering time for understanding, auditing and perhaps integrating the thing, even before deploying it. The bigger the set of useless features on a project, the bigger the risk in introducing an unknown into the critical paths that allow you to ship/provide a product/service. While many global "base" technologies like Postgres, enterprise Linux, Redis, NodeJS, Tomecat, the JDKs, etc are solid lighthouse projects with better quality than what "just writing your own" without world-class engineering teams and investment could produce in a reasonable amount of time, many others are just not that good. The cost of "adopting" a 2nd- or 3rd-rate FOSS solution may be very high and introduce significant technical debt down the road. In way to many companies, engineers (including engineering managers) have to explain to a non-technical C suite that OSS/FOSS IS NOT FREE and NOT GUARANTEED TO BE CHEAPER than going with proprietary or self-written, minimal, tested and concise solutions in any given context. They can be, of course, and with some engineering investment they can be for the longest time in the feature, but every case is different and a "wave" certainly is not what just removes the need to do this kind of evaluation.

stilisstuk 6y ago

Niskanen and budget maximizing https://en.m.wikipedia.org/wiki/Budget-maximizing_model https://en.m.wikipedia.org/wiki/Budget-maximizing_model Old School political science I have more faith in bureau shaping, which is almost the same.https://en.m.wikipedia.org/wiki/Bureau-shaping_mode https://en.m.wikipedia.org/wiki/Bureau-shaping_mode My it dep is mostly run by an Oss linux geek. This is super cool

dv_dt 6y ago

I think this is true in some cases, but there are others where an engineering org is proposing building an internal solution on an open source stack vs fewer dedicated internal resources with a proprietary solution and the argument becomes one of using internal resources to do something which is maybe a little outside of "core competency" areas vs a proprietary solution that may claim to do everything needed with a risk of limiting options either technically or financially in the future. There I've seen a pattern of short term financial cost overriding future risk, but maybe that shifts a little due to pandemic outcomes.