13 ms·

Data behind high-functioning engineering organizations [pdf]

144 points by rnjailamba 4y ago

rnjailamba 4y ago
https://www.iconiqcapital.com/growth/insights/2022-engineering-report https://www.iconiqcapital.com/growth/insights/2022-engineeri...
jugg1es 4y ago
wow 89% use Atlassian tools..
- jimbob1911 4y ago
  Aka 89% are clueless and just pay for the next shiny tool that has no real value
  - manv1 4y ago
    If you think atlassian tools have no value you haven't been in software development for long. They might suck, but they suck substantially less than anything else.
    - deleted 4y ago
      [deleted]
    - Aperocky 4y ago
      They suck because they attempt to add feature to something that doesn't need more features. They can probably be a small, healthy org but they insist on this cloud "solution". Now their customer are leaving them and they are paying so much more for all the extra fluff. Well deserved loss.
      - splittingTimes 4y ago
        Where do the customers leave to? We are currently giving up our self hosted Redmine instance as nobody is willing to take ownership on maintenance. We switch to Jira cloud.
        john_the_writer 4y ago
        github projects does the same job.
  - tstrimple 4y ago
    I wouldn't exactly call Jira or Confluence shiny.
  - plaguepilled 4y ago
    Atlassian tools are extremely boring but maintain a core set of features that many projects use. Their vanilla status is bordering on Microsoft Office levels of drudgery. What on earth are you comparing it to?
leetrout 4y ago
> Developer productivity can be compared to a sales funnel, with key metrics that can be tracked at each stage I dont like the idea of ruthless efficiency by tracking metrics similar to a sales funnel. It gives me the mental image of everyone competing and inflating numbers or gaming the system. https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- xmorg 4y ago
  Working with the iconiq capacity allocation framework was just awful. It was really frustrating because more time was spent about how work should be coded than the work itself. Quality didn't end up mattering because there was no accountability for actually delivering good software because following iconiq's standards was considered good enough. It was weird because management would routinely be surprised that our metrics were good but reality was not. As a grunt, it was frustrating because there was no room for productivity improvements because those were to only come from the top.
- Silhouette 4y ago
  Managers have become obsessed with being "agile" and "data driven" and yet never seem to notice how much useful engineering time and attention is being lost to all of those finely detailed breakdowns and regular meetings and tracking and monitoring processes. A common failure mode now seems to be that managers focus on metrics that don't really represent any useful reality to begin with but persist in trying to optimise those metrics even when their engineering team is telling them plainly that they're looking at garbage in and garbage out. The one metric no-one in management ever seems to consider is how much all of the management overheads themselves cost.
  - andyjsong 4y ago
    There is a lack of mutual understanding that shipping code and closing a deal have a lot of similarities. A Pull Request is basically the same as an Opportunity in your JIRA/SFDC. We can estimate when the PR will be pushed to prod or the contract gets signed. Generally, the more experience you have, the better you’re at setting expectations on when a PR/opp will close. The PR or opp can stall, but it’s up to the person responsible for the PR or opp to notify the rest of the team that there will be a delay or figure out a way to unblock. The big difference is the lack of accountability. AFAIK if a developer misses shipping during a sprint, they’ll just add it to the next sprint. If a salesperson does this, after 2 quarters of missing your quota, you’re fired. Doesn’t matter if you had monster quarters before that, if you’re not hitting your numbers, GTFO. I think this is because sales have very clear metrics that everyone can agree upon: revenue, number of demos, cold outreach, etc. We’ve been tracking these types of metrics in cuneiform. Comp sci wasn’t a thing until the 1960s. It’s still too new and everyone has an opinion on what metrics to focus on without angering a developer for micromanaging them. This is where the friction lies; until that’s fixed, you will not have a cohesive team. I hope that AI takes over and we don’t need devs or salespeople anymore. context: I’m a sales guy that sold software that tracks “engineering performance” to VPs of Eng and CTOs depending on the size of your company. It was shocking to see the level of variance in measuring performance metrics for eng teams.
birdhouse 4y ago
Browsed through the slides but didn’t encounter any lightbulb moments or any specific enlightenment. What are the key takeaways or lessons here?
- smeagull 4y ago
  Fill slides with figures and paragraphs of text and people will pay you lots of money to do the work of their existing teams.
jeffbee 4y ago
What aspect of the surveyed companies suggests that they are "high functioning"?
- tristanstr 4y ago
  FTX is on the list. Gotta be on to something.
  - TrickyRick 4y ago
    That you too can have a highly functional Ponzi scheme if you follow these practices?
    - jeffbee 4y ago
      Would love to see FTX's Jira admitted to evidence.
      - AnimalMuppet 4y ago
        I chuckled at this. Then I became mildly terrified. Please never mention that anywhere a lawyer can hear.
      - TrickyRick 4y ago
        The press made a big thing of them using "emojis in a chat app" (Presumably Slack) for approving expenses. I wonder if they actually used Jira?
dagss 4y ago
"Most organizations have shifted to organize their engineering teams by product" We have done the same. The great thing about it is locating decision makers and customer insight close to developers. The bad thing is that what is perceived to be a "product" can be very different from the natural division of technical work to be done. I sometimes joke that our Head of Product ended up involuntarily being our Chief Architect. The master plan of the tech has to be divided along these fault lines between Product A and Product B. In Sales these may be sold as separate products. But the reality is that 80% of functio ality is shared between them, and these 80% do not get a good organization developed around it, since the A vs B split is at the core of the company chart.
- kqr 4y ago
  Would it work to denote that 80 % as a separate product X that only has internal customers? On the one hand, it's the obvious solution. On the other hand, that would move the developers of X further away from the customers of A and B.
  - silasdavis 4y ago
    Also the internal product is, in effect, a compulsory purchase by teams A and B, therein lacking the selection pressure that normally drives product development. What other approaches to shared-something cross functional product teams are there?
xedrac 4y ago
Perhaps its my disdain for metrics and "efficiency process" in general, but skimming through these slides made me want to run for the hills. A high functioning team, in my experience, is one that has clear direction, autonomy, communicates well, and has enough trust to operate as efficiently as possible to meet the company's needs.
- greesil 4y ago
  I have been on projects that go nowhere when started, but as soon as the politics are settled down and everyone is rowing in the same direction productivity is unlocked. Then, a reorg happens. But yes, please measure how many lines of code I write.
  - leetrout 4y ago
    Bingo. This has been my experience as well. It's like once you reach a certain level of management mirrors don't exist and problems only exist elsewhere in the org.
- osigurdson 4y ago
  Agree. This feels like 90s era buffoonery. Slick slides with nice pantone selection and gradients. Entirely based on data, yet no data is ever provided - because confidentiality. Jabba says is not how science works.
pdyc 4y ago
Shameless plug: In case you want to get similar data for your company and you are using github for issues management, please give my product https://app.gitimprove.com https://app.gitimprove.com a try its in alpha right now. Website is https://gitimprove.com https://gitimprove.com.
- abatilo 4y ago
  Good luck with your product! Hope you have some great success
dnadler 4y ago
Does anyone have experience working within a matrix team? It seems like an interesting idea.
- TrickyRick 4y ago
  Yep, it was a way of getting more managers to the same number of engineers. Granted it was only 1 org so maybe they were just bad, but I had one line manager and one (Or sometimes multiple) project managers. The project managers compete for engineer time, which is supposed to be managed by the line manager allocating you to different projects. In reality nobody wants to say no and it was very easy for project managers to lowball project estimates thereby squeezing people in on 2 or sometimes even 3 projects at the same time. "How you manage your time was up to you but the spreadsheet says these 3 projects will fill 40 hours a week, if it actually takes 100 hours maybe you're bad at time management" was a pretty common attitude from managers. I didn't stay long and neither did anyone else who had even the slightest talent. The company was a major telco in a European country, so not a small startup with growth pains but a proper enterprise org with several thousands employees.
  - dnadler 4y ago
    Ah, I can totally see how that can happen. Thanks for the insight!
TrickyRick 4y ago
> More engineering organizations are starting to track developer productivity, with the top metrics reported on being number of bugs, %of committed software, working software, and PR to release time Sounds like an absolute dystopia.
mr_tristan 4y ago
This doesn’t seem like they figured out anything about “high functioning teams”, they just performed a survey. Like, do these companies actually ship faster, have lower rates of issues, better MTTR, etc? This just says things like “there are more full stack engineers”. At least the Puppet state of DevOps report gives you a “low, medium, high” categorization. This doesn’t seem to be describing much other than some trends they saw this year.
- taeric 4y ago
  Learning is a lot more about observing and surveying than folks typically acknowledge. Indeed, one of the more dangerous things I notice, on a regular basis, is a heavy set of beliefs built up on reasoning without observation. They typically betray far more of the personal wants of the reasoner than they do of reality. Of course, that way is another strawman. One of only observations with no reasoning on top.

144 points by rnjailamba 4y ago

https://www.iconiqcapital.com/growth/insights/2022-engineering-report https://www.iconiqcapital.com/growth/insights/2022-engineeri...

jugg1es 4y ago

wow 89% use Atlassian tools..

jimbob1911 4y ago
Aka 89% are clueless and just pay for the next shiny tool that has no real value
- manv1 4y ago
  If you think atlassian tools have no value you haven't been in software development for long. They might suck, but they suck substantially less than anything else.
  - deleted 4y ago
    [deleted]
  - Aperocky 4y ago
    They suck because they attempt to add feature to something that doesn't need more features. They can probably be a small, healthy org but they insist on this cloud "solution". Now their customer are leaving them and they are paying so much more for all the extra fluff. Well deserved loss.
    - splittingTimes 4y ago
      Where do the customers leave to? We are currently giving up our self hosted Redmine instance as nobody is willing to take ownership on maintenance. We switch to Jira cloud.
      - john_the_writer 4y ago
        github projects does the same job.
- tstrimple 4y ago
  I wouldn't exactly call Jira or Confluence shiny.
- plaguepilled 4y ago
  Atlassian tools are extremely boring but maintain a core set of features that many projects use. Their vanilla status is bordering on Microsoft Office levels of drudgery. What on earth are you comparing it to?

leetrout 4y ago

> Developer productivity can be compared to a sales funnel, with key metrics that can be tracked at each stage I dont like the idea of ruthless efficiency by tracking metrics similar to a sales funnel. It gives me the mental image of everyone competing and inflating numbers or gaming the system. https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

xmorg 4y ago
Working with the iconiq capacity allocation framework was just awful. It was really frustrating because more time was spent about how work should be coded than the work itself. Quality didn't end up mattering because there was no accountability for actually delivering good software because following iconiq's standards was considered good enough. It was weird because management would routinely be surprised that our metrics were good but reality was not. As a grunt, it was frustrating because there was no room for productivity improvements because those were to only come from the top.
Silhouette 4y ago
Managers have become obsessed with being "agile" and "data driven" and yet never seem to notice how much useful engineering time and attention is being lost to all of those finely detailed breakdowns and regular meetings and tracking and monitoring processes. A common failure mode now seems to be that managers focus on metrics that don't really represent any useful reality to begin with but persist in trying to optimise those metrics even when their engineering team is telling them plainly that they're looking at garbage in and garbage out. The one metric no-one in management ever seems to consider is how much all of the management overheads themselves cost.
- andyjsong 4y ago
  There is a lack of mutual understanding that shipping code and closing a deal have a lot of similarities. A Pull Request is basically the same as an Opportunity in your JIRA/SFDC. We can estimate when the PR will be pushed to prod or the contract gets signed. Generally, the more experience you have, the better you’re at setting expectations on when a PR/opp will close. The PR or opp can stall, but it’s up to the person responsible for the PR or opp to notify the rest of the team that there will be a delay or figure out a way to unblock. The big difference is the lack of accountability. AFAIK if a developer misses shipping during a sprint, they’ll just add it to the next sprint. If a salesperson does this, after 2 quarters of missing your quota, you’re fired. Doesn’t matter if you had monster quarters before that, if you’re not hitting your numbers, GTFO. I think this is because sales have very clear metrics that everyone can agree upon: revenue, number of demos, cold outreach, etc. We’ve been tracking these types of metrics in cuneiform. Comp sci wasn’t a thing until the 1960s. It’s still too new and everyone has an opinion on what metrics to focus on without angering a developer for micromanaging them. This is where the friction lies; until that’s fixed, you will not have a cohesive team. I hope that AI takes over and we don’t need devs or salespeople anymore. context: I’m a sales guy that sold software that tracks “engineering performance” to VPs of Eng and CTOs depending on the size of your company. It was shocking to see the level of variance in measuring performance metrics for eng teams.

birdhouse 4y ago

Browsed through the slides but didn’t encounter any lightbulb moments or any specific enlightenment. What are the key takeaways or lessons here?

smeagull 4y ago
Fill slides with figures and paragraphs of text and people will pay you lots of money to do the work of their existing teams.

jeffbee 4y ago

What aspect of the surveyed companies suggests that they are "high functioning"?

tristanstr 4y ago
FTX is on the list. Gotta be on to something.
- TrickyRick 4y ago
  That you too can have a highly functional Ponzi scheme if you follow these practices?
  - jeffbee 4y ago
    Would love to see FTX's Jira admitted to evidence.
    - AnimalMuppet 4y ago
      I chuckled at this. Then I became mildly terrified. Please never mention that anywhere a lawyer can hear.
    - TrickyRick 4y ago
      The press made a big thing of them using "emojis in a chat app" (Presumably Slack) for approving expenses. I wonder if they actually used Jira?

dagss 4y ago

"Most organizations have shifted to organize their engineering teams by product" We have done the same. The great thing about it is locating decision makers and customer insight close to developers. The bad thing is that what is perceived to be a "product" can be very different from the natural division of technical work to be done. I sometimes joke that our Head of Product ended up involuntarily being our Chief Architect. The master plan of the tech has to be divided along these fault lines between Product A and Product B. In Sales these may be sold as separate products. But the reality is that 80% of functio ality is shared between them, and these 80% do not get a good organization developed around it, since the A vs B split is at the core of the company chart.

kqr 4y ago
Would it work to denote that 80 % as a separate product X that only has internal customers? On the one hand, it's the obvious solution. On the other hand, that would move the developers of X further away from the customers of A and B.
- silasdavis 4y ago
  Also the internal product is, in effect, a compulsory purchase by teams A and B, therein lacking the selection pressure that normally drives product development. What other approaches to shared-something cross functional product teams are there?

xedrac 4y ago

Perhaps its my disdain for metrics and "efficiency process" in general, but skimming through these slides made me want to run for the hills. A high functioning team, in my experience, is one that has clear direction, autonomy, communicates well, and has enough trust to operate as efficiently as possible to meet the company's needs.

greesil 4y ago
I have been on projects that go nowhere when started, but as soon as the politics are settled down and everyone is rowing in the same direction productivity is unlocked. Then, a reorg happens. But yes, please measure how many lines of code I write.
- leetrout 4y ago
  Bingo. This has been my experience as well. It's like once you reach a certain level of management mirrors don't exist and problems only exist elsewhere in the org.
osigurdson 4y ago
Agree. This feels like 90s era buffoonery. Slick slides with nice pantone selection and gradients. Entirely based on data, yet no data is ever provided - because confidentiality. Jabba says is not how science works.

pdyc 4y ago

Shameless plug: In case you want to get similar data for your company and you are using github for issues management, please give my product https://app.gitimprove.com https://app.gitimprove.com a try its in alpha right now. Website is https://gitimprove.com https://gitimprove.com.

abatilo 4y ago
Good luck with your product! Hope you have some great success

dnadler 4y ago

Does anyone have experience working within a matrix team? It seems like an interesting idea.

TrickyRick 4y ago
Yep, it was a way of getting more managers to the same number of engineers. Granted it was only 1 org so maybe they were just bad, but I had one line manager and one (Or sometimes multiple) project managers. The project managers compete for engineer time, which is supposed to be managed by the line manager allocating you to different projects. In reality nobody wants to say no and it was very easy for project managers to lowball project estimates thereby squeezing people in on 2 or sometimes even 3 projects at the same time. "How you manage your time was up to you but the spreadsheet says these 3 projects will fill 40 hours a week, if it actually takes 100 hours maybe you're bad at time management" was a pretty common attitude from managers. I didn't stay long and neither did anyone else who had even the slightest talent. The company was a major telco in a European country, so not a small startup with growth pains but a proper enterprise org with several thousands employees.
- dnadler 4y ago
  Ah, I can totally see how that can happen. Thanks for the insight!

TrickyRick 4y ago

> More engineering organizations are starting to track developer productivity, with the top metrics reported on being number of bugs, %of committed software, working software, and PR to release time Sounds like an absolute dystopia.

mr_tristan 4y ago

This doesn’t seem like they figured out anything about “high functioning teams”, they just performed a survey. Like, do these companies actually ship faster, have lower rates of issues, better MTTR, etc? This just says things like “there are more full stack engineers”. At least the Puppet state of DevOps report gives you a “low, medium, high” categorization. This doesn’t seem to be describing much other than some trends they saw this year.

taeric 4y ago
Learning is a lot more about observing and surveying than folks typically acknowledge. Indeed, one of the more dangerous things I notice, on a regular basis, is a heavy set of beliefs built up on reasoning without observation. They typically betray far more of the personal wants of the reasoner than they do of reality. Of course, that way is another strawman. One of only observations with no reasoning on top.