6 ms·

The phrase thrown around was “collaboration headwind”, the idea was if project success depends on 1 person with a 95% chance of success, project success also ha

The phrase thrown around was “collaboration headwind”, the idea was if project success depends on 1 person with a 95% chance of success, project success also had a 95% chance. But if 10 people each need to succeed at a 95% chance, suddenly the project success likelihood becomes 60%… In reality, lazy domain owners layered on processes, meetings, documents, and multiple approvals until it took 6 months to change the text on a button, ugh

pclmulqdq 1y ago
There's a culture of "I won't approve this unless it does something for me" at Google. So now changing the text on a button comes with 2 minor refactors, 10 obvious-but-ignored bugfixes, and 5 experiments that it is actually better.
- moregrist 1y ago
  While this sounds pretty frustrating, there is at least a small upside: at least you get to the obvious-but-ignored bugfixes. Most smaller places don’t have the bandwidth and many larger ones don’t have the desire. I’m not sure if that makes up for bugs potentially introduced in the refactors, though.
  - dietr1ch 1y ago
    Well, when the owner asks for a whole test suite that didn't exist to get a fix in, what most likely happens is that you just wasted your time in a draft CL that will get lost.
    - vkou 1y ago
      They aren't asking for you to write tests because 'it benefits them', they are asking you to write tests because as a professional engineer, you should write tests, and not just yolo it. Look, sometimes you may have good reasons for why a test is impractical. You are allowed to push back, or look for a different reviewer. There's a hundred thousand people in the firm, you should be able to find one or two that will let you submit literally anything that compiles. But most of the time, the reviewer is giving you advice that you should take.
      - pclmulqdq 1y ago
        If you are turning a button to a slightly different shade of blue and it's not a button you own, the owner of the button should not be asking you to write tests for the button.
        vkou 1y ago
        If the case is as simple as you describe, surely there's more than one owner of the code that can approve this, if one guy is being unreasonable. If there is actually only one owner that can approve changes to the package, there's something really weird and unusual about that project setup, or it's someone's internal hobby project that they wrote five years ago and semi-maintain, in which case, I have to wonder why you submitting one-liner changes to it is all that important. We're all adults, we all work together, we can all work this out. If someone absolutely insists on being an asshole, escalate. It's why you have a manager, and why they have a manager. My experience is that very few people are unreasonable assholes. There's always plenty of organizational, vision, strategy, and execution problem in any billion-dollar company, but 'people are unreasonable in code reviews' is not one I'd put in the top 10. It might be something that ruins your day once or twice, but that doesn't make it systemic.
        dietr1ch 1y ago
        > If someone absolutely insists on being an asshole, escalate. That's doubling down on time spent on contributing back. It's usually cheaper to workaround the issue once you notice it'll be way harder than it should be (not hard at all).
        econ 1y ago
        I would think the person is more interesting and more relevant than the button. One doesn't create hurdles when I'm trying to work. You just don't do it. I've allowed people to build a whole obstacle course one time. Decades later it stil has me randomly burst out in laughter. It's like hoarding technical debt until nothing in the code base makes sense or even looks familiar. You just don't do that...
    - specialist 1y ago
      Do you mean the relevant code area(s) didn't have (sufficient) tests? You're being asked to backfill those missing tests in addition to your fix?
      - dietr1ch 1y ago
        Yes. I've experienced pushback from obvious fixes with requests to formally test their code for the first time. All because it may break someone. Even when I presented a real defect based on docs/comments and fixed it. You'd think that if they truly cared about breakages they'd already have some tests for it from where I can easily start.
        vkou 1y ago
        > All because it may break someone. Even when I presented a real defect based on docs/comments and fixed it. It's great that you found a bug and fixed it. The problem is, how do you know that there are no other regressions? Code is a liability. Once you check it in, the team that owns it is responsible for it. Untested code is a liability of unknown scope. It's quite understandable why they don't want to accept someone's contributions, when the contributor isn't the one who will ultimately be dealing with any of the consequences. If you think they are being mean and lazy, imagine if the tables were reversed. I don't accept puppies or elephants as gifts for similar reasons. It's unfortunate that existing test coverage sucks. In this case, the best way forward should be for the team in question to improve coverage, and for you to then submit your fix + a test for it. And if they don't have budget to do this, then that sucks, but that's their call to make, and that's a signal that the project in question is abandonware. And it's fine for a large company to have a bunch of abandonware. If it works, and produces value, the optimal amount of ongoing development effort to invest into a piece of software may, depending on the circumstances, be near-zero.
        __turbobrew__ 1y ago
        I don’t think that is necessarily unreasonable. The team may have the same constraints on themselves in that they wouldn’t touch the code either until tests are written.
        1y ago
apercu 1y ago
> lazy domain owners Interesting. As a consultant for the most of the last 25 years, my experience is the domain owners are typically invested and have strong opinions on the things that impact their jobs. Executive leadership, on the other hand, doesn't want to actually know the issues and eyes glaze over as they look at their watches because they have a tee time.
sollewitt 1y ago
Coordination Headwind: https://komoroske.com/slime-mold/ https://komoroske.com/slime-mold/
- repeekad 1y ago
  Thanks, this may be the name they used it’s been a little while
steveBK123 1y ago
The assumptions in that math are wrong anyway. Once you depend on 10 people, the chance that they each achieve "95% successful execution" is 0. This is only partially down to the impossibility of having every staff member on a project be A++ players. There is coordination RISK not just coordination overhead. Think planning a 2 week trip with your spouse with multiple planes/trains/hotels, museum/exhibit ticket bookings, meal reservations, etc. Inevitably something gets misunderstood/miscommunicated between the two of you and therefore mis-implemented. Now add more communication nodes to the graph and watch the error rate explode.
- nostrademons 1y ago
  That's what the math is reflecting. Project succeeds if all of 10 people does their job well. Each person has a 95% chance of succeeding. 0.95^10 ~= 60%, and so the chance that all 10 people do their job successfully is ~60%. Those jobs also include things like management and product design, and so the coordination risk is reflected in the 5% chance that the manager drops the ball on communication. (As a manager, I suspect that chance is significantly more than 5% and that's why overall success rates are even lower.)
  - steveBK123 1y ago
    That's what I mean "only 5%" encapsulating all failure modes (comms/implementation/coordination/etc) is very low. And that under-estimation compounds to make the top level 60% much higher than it should be. A 7.5% rate takes top-level success odds below 50% - 46%. A not unrealistic 10% takes the top level down to 35%. Etc.
    - econ 1y ago
      For extra fun we are doing more than one project in a row.
xnx 1y ago
The old "If you want to go fast, go alone. If you want to go far, go together."
- nostrademons 1y ago
  Also why the optimal business strategy seems to be to go as far as you can alone and then bring on other people when you're running out of steam.
miki123211 1y ago
Another side of this coin is that the expected payoff from a project depends on how many unrelated projects your organization is engaging in, which is deeply counterintuitive to most people. Every project carries with it three possibilities: that of success, where the company makes money, that of failure, where the company does not, and that of a "critical failure", where the project goes so wrong that it results in a major lawsuit, regulatory fine or PR disaster that costs the company more than the project was ever expected to make. If you're a startup, the worst that can happen to your company is the value going to 0. From an investor's perspective, there's not much of a difference between burning all the money ($10m) and not finding product-market-fit (normal failure), or your company getting sued for $3b and going bankrupt (critical failure). The result is the same, the investment is lost. For a large corporation, a $3b lawsuit is far more costly than sinking $10m into a failed project. You can trade off these three possibilities against each other. Maybe forcing each release through an arduous checklist of legal review or internationalization and accessibility testing decreases success rates by 10%, but moves the "critical failure rate" from 1% to 0.5%. From a startup's perspective, this is a bad tradeoff, but if you're a barely-profitable R&D project at big co, the checklist is the right call to make. This problem is independent from all the other causes to which bureaucracy is usually attributed, like the number of layers of management, internal culture, or "organizational scar tissue." Just from a legal and brand safety perspective, the bigger your org, the more bureaucracy makes sense, no matter how efficient you can get your org to be.
- kridsdale3 1y ago
  This is an excellent explanation of the culture of BigTech.
- YZF 1y ago
  I like the ideas here but I think the actual chance of getting sued for $3b is so small as to be negligible in the context of costs. It's also questionable how much the additional process/overhead moves the needle on that chance. Larger companies also have various "shields" against these sorts of lawsuits. E.g. they lobby politicians, they employ lawyers, they have legal and IP protection. Just like anything else in life you want to look at the present value and then get insurance for huge risks. That said agree that a startup can take more risk but I don't think that is the major factor explaining why larger companies tend to be process heavy and slower.
zenogantner 1y ago
Well, good management/tech leadership is about making sure that the risks coming from individual failure points (10 people in your example) are recognized and mitigated, and that the individuals involved can flag risks and conflicts early enough so that the overall project success probability does not go down as you describe...

pclmulqdq 1y ago

There's a culture of "I won't approve this unless it does something for me" at Google. So now changing the text on a button comes with 2 minor refactors, 10 obvious-but-ignored bugfixes, and 5 experiments that it is actually better.

moregrist 1y ago
While this sounds pretty frustrating, there is at least a small upside: at least you get to the obvious-but-ignored bugfixes. Most smaller places don’t have the bandwidth and many larger ones don’t have the desire. I’m not sure if that makes up for bugs potentially introduced in the refactors, though.
- dietr1ch 1y ago
  Well, when the owner asks for a whole test suite that didn't exist to get a fix in, what most likely happens is that you just wasted your time in a draft CL that will get lost.
  - vkou 1y ago
    They aren't asking for you to write tests because 'it benefits them', they are asking you to write tests because as a professional engineer, you should write tests, and not just yolo it. Look, sometimes you may have good reasons for why a test is impractical. You are allowed to push back, or look for a different reviewer. There's a hundred thousand people in the firm, you should be able to find one or two that will let you submit literally anything that compiles. But most of the time, the reviewer is giving you advice that you should take.
    - pclmulqdq 1y ago
      If you are turning a button to a slightly different shade of blue and it's not a button you own, the owner of the button should not be asking you to write tests for the button.
      - vkou 1y ago
        If the case is as simple as you describe, surely there's more than one owner of the code that can approve this, if one guy is being unreasonable. If there is actually only one owner that can approve changes to the package, there's something really weird and unusual about that project setup, or it's someone's internal hobby project that they wrote five years ago and semi-maintain, in which case, I have to wonder why you submitting one-liner changes to it is all that important. We're all adults, we all work together, we can all work this out. If someone absolutely insists on being an asshole, escalate. It's why you have a manager, and why they have a manager. My experience is that very few people are unreasonable assholes. There's always plenty of organizational, vision, strategy, and execution problem in any billion-dollar company, but 'people are unreasonable in code reviews' is not one I'd put in the top 10. It might be something that ruins your day once or twice, but that doesn't make it systemic.
        dietr1ch 1y ago
        > If someone absolutely insists on being an asshole, escalate. That's doubling down on time spent on contributing back. It's usually cheaper to workaround the issue once you notice it'll be way harder than it should be (not hard at all).
        econ 1y ago
        I would think the person is more interesting and more relevant than the button. One doesn't create hurdles when I'm trying to work. You just don't do it. I've allowed people to build a whole obstacle course one time. Decades later it stil has me randomly burst out in laughter. It's like hoarding technical debt until nothing in the code base makes sense or even looks familiar. You just don't do that...
  - specialist 1y ago
    Do you mean the relevant code area(s) didn't have (sufficient) tests? You're being asked to backfill those missing tests in addition to your fix?
    - dietr1ch 1y ago
      Yes. I've experienced pushback from obvious fixes with requests to formally test their code for the first time. All because it may break someone. Even when I presented a real defect based on docs/comments and fixed it. You'd think that if they truly cared about breakages they'd already have some tests for it from where I can easily start.
      - vkou 1y ago
        > All because it may break someone. Even when I presented a real defect based on docs/comments and fixed it. It's great that you found a bug and fixed it. The problem is, how do you know that there are no other regressions? Code is a liability. Once you check it in, the team that owns it is responsible for it. Untested code is a liability of unknown scope. It's quite understandable why they don't want to accept someone's contributions, when the contributor isn't the one who will ultimately be dealing with any of the consequences. If you think they are being mean and lazy, imagine if the tables were reversed. I don't accept puppies or elephants as gifts for similar reasons. It's unfortunate that existing test coverage sucks. In this case, the best way forward should be for the team in question to improve coverage, and for you to then submit your fix + a test for it. And if they don't have budget to do this, then that sucks, but that's their call to make, and that's a signal that the project in question is abandonware. And it's fine for a large company to have a bunch of abandonware. If it works, and produces value, the optimal amount of ongoing development effort to invest into a piece of software may, depending on the circumstances, be near-zero.
      - __turbobrew__ 1y ago
        I don’t think that is necessarily unreasonable. The team may have the same constraints on themselves in that they wouldn’t touch the code either until tests are written.
      - 1y ago

apercu 1y ago

> lazy domain owners Interesting. As a consultant for the most of the last 25 years, my experience is the domain owners are typically invested and have strong opinions on the things that impact their jobs. Executive leadership, on the other hand, doesn't want to actually know the issues and eyes glaze over as they look at their watches because they have a tee time.

sollewitt 1y ago

Coordination Headwind: https://komoroske.com/slime-mold/ https://komoroske.com/slime-mold/

repeekad 1y ago
Thanks, this may be the name they used it’s been a little while

steveBK123 1y ago

The assumptions in that math are wrong anyway. Once you depend on 10 people, the chance that they each achieve "95% successful execution" is 0. This is only partially down to the impossibility of having every staff member on a project be A++ players. There is coordination RISK not just coordination overhead. Think planning a 2 week trip with your spouse with multiple planes/trains/hotels, museum/exhibit ticket bookings, meal reservations, etc. Inevitably something gets misunderstood/miscommunicated between the two of you and therefore mis-implemented. Now add more communication nodes to the graph and watch the error rate explode.

nostrademons 1y ago
That's what the math is reflecting. Project succeeds if all of 10 people does their job well. Each person has a 95% chance of succeeding. 0.95^10 ~= 60%, and so the chance that all 10 people do their job successfully is ~60%. Those jobs also include things like management and product design, and so the coordination risk is reflected in the 5% chance that the manager drops the ball on communication. (As a manager, I suspect that chance is significantly more than 5% and that's why overall success rates are even lower.)
- steveBK123 1y ago
  That's what I mean "only 5%" encapsulating all failure modes (comms/implementation/coordination/etc) is very low. And that under-estimation compounds to make the top level 60% much higher than it should be. A 7.5% rate takes top-level success odds below 50% - 46%. A not unrealistic 10% takes the top level down to 35%. Etc.
  - econ 1y ago
    For extra fun we are doing more than one project in a row.

xnx 1y ago

The old "If you want to go fast, go alone. If you want to go far, go together."

nostrademons 1y ago
Also why the optimal business strategy seems to be to go as far as you can alone and then bring on other people when you're running out of steam.

miki123211 1y ago

Another side of this coin is that the expected payoff from a project depends on how many unrelated projects your organization is engaging in, which is deeply counterintuitive to most people. Every project carries with it three possibilities: that of success, where the company makes money, that of failure, where the company does not, and that of a "critical failure", where the project goes so wrong that it results in a major lawsuit, regulatory fine or PR disaster that costs the company more than the project was ever expected to make. If you're a startup, the worst that can happen to your company is the value going to 0. From an investor's perspective, there's not much of a difference between burning all the money ($10m) and not finding product-market-fit (normal failure), or your company getting sued for $3b and going bankrupt (critical failure). The result is the same, the investment is lost. For a large corporation, a $3b lawsuit is far more costly than sinking $10m into a failed project. You can trade off these three possibilities against each other. Maybe forcing each release through an arduous checklist of legal review or internationalization and accessibility testing decreases success rates by 10%, but moves the "critical failure rate" from 1% to 0.5%. From a startup's perspective, this is a bad tradeoff, but if you're a barely-profitable R&D project at big co, the checklist is the right call to make. This problem is independent from all the other causes to which bureaucracy is usually attributed, like the number of layers of management, internal culture, or "organizational scar tissue." Just from a legal and brand safety perspective, the bigger your org, the more bureaucracy makes sense, no matter how efficient you can get your org to be.

kridsdale3 1y ago
This is an excellent explanation of the culture of BigTech.
YZF 1y ago
I like the ideas here but I think the actual chance of getting sued for $3b is so small as to be negligible in the context of costs. It's also questionable how much the additional process/overhead moves the needle on that chance. Larger companies also have various "shields" against these sorts of lawsuits. E.g. they lobby politicians, they employ lawyers, they have legal and IP protection. Just like anything else in life you want to look at the present value and then get insurance for huge risks. That said agree that a startup can take more risk but I don't think that is the major factor explaining why larger companies tend to be process heavy and slower.

zenogantner 1y ago

Well, good management/tech leadership is about making sure that the risks coming from individual failure points (10 people in your example) are recognized and mitigated, and that the individuals involved can flag risks and conflicts early enough so that the overall project success probability does not go down as you describe...