5 ms·

Big corporations are extremely inefficient. Why does everyone want to copy them? And why do we use all the bulky tools they produce when there are far simpler,

Big corporations are extremely inefficient. Why does everyone want to copy them? And why do we use all the bulky tools they produce when there are far simpler, better and more open alternatives available?

beagle3 7y ago
Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?
- unixhero 7y ago
  Proper secure development lifecycle approach for all software projects producing code?
  - beagle3 7y ago
    What are those? There are tens of such approaches, many of them practiced at large corps, and in my experience almost none of them deliver the supposed benefits, with few exceptions (NASA, MISRA) that are cost prohibitive except for very narrow fields. And what does that have to do with “copying large corps” and/or monorepo vs multirepo?
- cryptica 7y ago
  I think it's because corporations treat developers like untrustworthy and replaceable cogs in a big machine - IMO, this is what's wrong, not multi-repos. If corporations gave their developers responsibility and ownership over the maintenance of specific repos which are heavily depended upon, then multi-repos would not be a problem. It's a great source of pride for developers to feel responsible for something and to take care of upstream developers. About simpler alternatives, non-corporate tools and libraries are often much better than corporate-built ones. For example, React was built by Facebook and is inferior to VueJS which was built independently yet React is much more popular. On top of this, I contend that if Evan You (the creator of VueJS) hadn't worked at Google, very few people would be using VueJS today; but the popularity has more to do with public image than technical merit. VueJS is associated with Google even though it was written independently; something so simple and elegant (especially at the time it came out) could only have been written by an indie dev. Too many cooks do spoil the broth unfortunately.
  - beagle3 7y ago
    I disagree about the nature of the problem. You can’t give sole ownership of a project people depend on to just one person - a bus factor of 1 is unacceptable in most circumstances. Busses happen, people leave, etc. Some things are also too big for one person. And when you give it to a team of proud capable developers, you will get ego wars, blame throwing, etc. with probability that increases quickly with team size. And that’s with capable, proud, responsible, cooperative people. But not all your people will be like that all the time - some will be burnet our, some have no passion and just try to do the minimum to not get fired. Some are sociopaths who actively try to shift work to others and only pride themselves on work they managed to avoid. Mom and pop software shops can afford non-replaceable-cogs, Google can’t (and afaik didn’t even in the old days when they were run by engineers and not MBAs). And this whole discussion of copying corps was in the context of mono Vs multi repo.... (For the record, VueJS and Evan You are awesome)
    - ratww 7y ago
      Giving ownership to a single person doesn't imply a bus factor of one. As long as someone can take over, it's not a problem. I've had this happen lots of times in my career, and most of the time dealing with such projects was faster than onboarding in a company-owned repo by at least an order of magnitude. I really mean it: minutes instead of days to be productive. In my experience most engineers have the best intentions and want to do the best work possible, until you force them into working in a bad environment where he's treated like... a cog in the machine with zero ability to influence his own productivity, happiness or even the quality of his code. The biggest reason large companies "can't afford" those things is because they're hell bent on turning engineerings into replaceable cogs, out of ideology.
      - beagle3 7y ago
        > In my experience most engineers have the best intentions and want to do the best work possible I agree, most of them do - but I've also been in a place that highers the best of the best, and despite the best intentions, some people cannot put their ego aside and the overall result is between less-than-ideal and downright-horrible
- hocuspocus 7y ago
  > Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. How do you measure that exactly?
  - beagle3 7y ago
    Personally, informal survey of people I know who work at big corps. Google and Microsoft have both evaluated this internally and reached that conclusion, with some of the evaluation criteria and conclusions publicly documented (though I don’t have links available on my phone, google will likely find them for you)
    - hocuspocus 7y ago
      Microsoft doesn't really use a monorepo. This leaves companies like Google and Facebook that made a decision fairly early in their existence, set constraints, and then spent several hundreds engineer-years into developing their own infrastructure and tools to support that decision. Does it work? Yes. Is it more efficient than what other companies of similar scale are doing? I don't believe anyone knows.
      - beagle3 7y ago
        From reading articles, I got the impression Windows is one monorepo (but it doesn’t include Office) and e.g. Visual Stduio is another. So it’s true that Microsoft doesn’t use a monorepo as a whole, but e.g. just Windows itself is a codebase significantly larger and more diverse than 99% of companies would have, and it does have multiple “independents” products - Notepad is an independent module from Solitaire and they are both independent of essentially everything else, whereas e.g. File Explorer, the win32 subsystem and the kernel often need to be updated together for new features. GoogBook made it early, but re-evaluated. For sure, they found the cost of switching is higher than any potential benefit. I must say that I have not found anyone describing benefits other than “git is faster for smaller repositories” (which was very true but is only slightly true with sparse checkouts and shallow branches these days) and “I like it that way”. Yosefk has a convincing article about “if your culture is bad, it doesn’t matter; if your culture is good, monorepos are technically easier until you hit some scale wall but then they don’t become harder that multirepo at that scale” IIRc. But w.r.t Microsoft I admit I’m not very well informed, thank you for setting the record straight.
- ex_amazon_sde 7y ago
  > it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos Amazon disagrees.
  - sokoloff 7y ago
    Does Amazon optimize for efficiency in software engineering, or just for the overall quantity of output? My sense, having spoken with Amazon SWEs and a fair number of engineering leaders is that they optimize for overall output, but not for efficiency (output per worker).
    - beagle3 7y ago
      Well, the measure of efficiency most of the world cares about is “overall profit”, which — around the efficient frontier — has a good proxy with “overall output”. I feel your “output per worker” measure of average individual efficiency is not a good metric to optimize - just fire everyone except your best employee and you’ve maximized it. Edit: employer -> employee typo
      - sokoloff 7y ago
        I think your first point is good and agree with it. Even within optimizing for that (rather than output/worker or output/salary), I think it's fair to examine whether specific choices are optimized for the same case that you are facing. Not everything done by very successful companies should be replicated in your business.

beagle3 7y ago

Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. What are those far simpler, better, more open alternatives?

unixhero 7y ago
Proper secure development lifecycle approach for all software projects producing code?
- beagle3 7y ago
  What are those? There are tens of such approaches, many of them practiced at large corps, and in my experience almost none of them deliver the supposed benefits, with few exceptions (NASA, MISRA) that are cost prohibitive except for very narrow fields. And what does that have to do with “copying large corps” and/or monorepo vs multirepo?
cryptica 7y ago
I think it's because corporations treat developers like untrustworthy and replaceable cogs in a big machine - IMO, this is what's wrong, not multi-repos. If corporations gave their developers responsibility and ownership over the maintenance of specific repos which are heavily depended upon, then multi-repos would not be a problem. It's a great source of pride for developers to feel responsible for something and to take care of upstream developers. About simpler alternatives, non-corporate tools and libraries are often much better than corporate-built ones. For example, React was built by Facebook and is inferior to VueJS which was built independently yet React is much more popular. On top of this, I contend that if Evan You (the creator of VueJS) hadn't worked at Google, very few people would be using VueJS today; but the popularity has more to do with public image than technical merit. VueJS is associated with Google even though it was written independently; something so simple and elegant (especially at the time it came out) could only have been written by an indie dev. Too many cooks do spoil the broth unfortunately.
- beagle3 7y ago
  I disagree about the nature of the problem. You can’t give sole ownership of a project people depend on to just one person - a bus factor of 1 is unacceptable in most circumstances. Busses happen, people leave, etc. Some things are also too big for one person. And when you give it to a team of proud capable developers, you will get ego wars, blame throwing, etc. with probability that increases quickly with team size. And that’s with capable, proud, responsible, cooperative people. But not all your people will be like that all the time - some will be burnet our, some have no passion and just try to do the minimum to not get fired. Some are sociopaths who actively try to shift work to others and only pride themselves on work they managed to avoid. Mom and pop software shops can afford non-replaceable-cogs, Google can’t (and afaik didn’t even in the old days when they were run by engineers and not MBAs). And this whole discussion of copying corps was in the context of mono Vs multi repo.... (For the record, VueJS and Evan You are awesome)
  - ratww 7y ago
    Giving ownership to a single person doesn't imply a bus factor of one. As long as someone can take over, it's not a problem. I've had this happen lots of times in my career, and most of the time dealing with such projects was faster than onboarding in a company-owned repo by at least an order of magnitude. I really mean it: minutes instead of days to be productive. In my experience most engineers have the best intentions and want to do the best work possible, until you force them into working in a bad environment where he's treated like... a cog in the machine with zero ability to influence his own productivity, happiness or even the quality of his code. The biggest reason large companies "can't afford" those things is because they're hell bent on turning engineerings into replaceable cogs, out of ideology.
    - beagle3 7y ago
      > In my experience most engineers have the best intentions and want to do the best work possible I agree, most of them do - but I've also been in a place that highers the best of the best, and despite the best intentions, some people cannot put their ego aside and the overall result is between less-than-ideal and downright-horrible
hocuspocus 7y ago
> Because it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos. How do you measure that exactly?
- beagle3 7y ago
  Personally, informal survey of people I know who work at big corps. Google and Microsoft have both evaluated this internally and reached that conclusion, with some of the evaluation criteria and conclusions publicly documented (though I don’t have links available on my phone, google will likely find them for you)
  - hocuspocus 7y ago
    Microsoft doesn't really use a monorepo. This leaves companies like Google and Facebook that made a decision fairly early in their existence, set constraints, and then spent several hundreds engineer-years into developing their own infrastructure and tools to support that decision. Does it work? Yes. Is it more efficient than what other companies of similar scale are doing? I don't believe anyone knows.
    - beagle3 7y ago
      From reading articles, I got the impression Windows is one monorepo (but it doesn’t include Office) and e.g. Visual Stduio is another. So it’s true that Microsoft doesn’t use a monorepo as a whole, but e.g. just Windows itself is a codebase significantly larger and more diverse than 99% of companies would have, and it does have multiple “independents” products - Notepad is an independent module from Solitaire and they are both independent of essentially everything else, whereas e.g. File Explorer, the win32 subsystem and the kernel often need to be updated together for new features. GoogBook made it early, but re-evaluated. For sure, they found the cost of switching is higher than any potential benefit. I must say that I have not found anyone describing benefits other than “git is faster for smaller repositories” (which was very true but is only slightly true with sparse checkouts and shallow branches these days) and “I like it that way”. Yosefk has a convincing article about “if your culture is bad, it doesn’t matter; if your culture is good, monorepos are technically easier until you hit some scale wall but then they don’t become harder that multirepo at that scale” IIRc. But w.r.t Microsoft I admit I’m not very well informed, thank you for setting the record straight.
ex_amazon_sde 7y ago
> it turns out big corporations that use multirepos tend to be even less efficient (about versioning) than those that use monorepos Amazon disagrees.
- sokoloff 7y ago
  Does Amazon optimize for efficiency in software engineering, or just for the overall quantity of output? My sense, having spoken with Amazon SWEs and a fair number of engineering leaders is that they optimize for overall output, but not for efficiency (output per worker).
  - beagle3 7y ago
    Well, the measure of efficiency most of the world cares about is “overall profit”, which — around the efficient frontier — has a good proxy with “overall output”. I feel your “output per worker” measure of average individual efficiency is not a good metric to optimize - just fire everyone except your best employee and you’ve maximized it. Edit: employer -> employee typo
    - sokoloff 7y ago
      I think your first point is good and agree with it. Even within optimizing for that (rather than output/worker or output/salary), I think it's fair to examine whether specific choices are optimized for the same case that you are facing. Not everything done by very successful companies should be replicated in your business.