8 ms·

When everything is important but nothing is getting done

389 points by goopthink 4y ago

bell-cot 4y ago
My gut reaction to the title of the post - fire 80% of the managers. Then announce skimpy buyouts for any employee who feels that they won't get enough face time with managers and/or time spent in meetings with the new manager/worker ratio.
lbriner 4y ago
Lots of really good points in here. I have worked at various companies who have struggled after the initial growth. Hubris can take over and we feel the need to over-extend, to sit at the big-boys table, to court the corporates with their fat (but very tight) wallets. I think a common cause is simply fear. There is a lot at stake when you have 50+ employees. Maybe those one or two markets are not big enough to support what we need any more so instead of razor focus and discipline which helps, we try and build everything for everyone and a lot of it is simply wasted effort. We also start having teams so instead of a design decision being taken by the entire management (and can get shouted down), it just gets done because "the design team" regardless of the value it is actually adding or not. In extremis, we see the FANG companies who have so many engineering staff, we can only hypothesize about what most of these people even do for 40 hours per week on massive salaries.
- bombcar 4y ago
  I wonder how much lost productivity comes from people saying "if only it had feature X" to salesmen to be polite instead of a simple "we're not interested and won't be buying".
  - goopthink 4y ago
    "The Mom Test" is an awesome look at that -- how bad questions and worse feedback leads to a lot of wasted effort. Highly recommend it. http://momtestbook.com/ http://momtestbook.com/
  - mason55 4y ago
    It's the difference between being a feature factory and having a real product strategy. If you actually have some kind of customer in mind, and you have ideas of problems that you're solving for them, then this can be valuable feedback even if it's a lie. Digging into the feature might expose you to a new customer problem that weren't aware of. You can talk to other customers and prospects and understand if this is a problem they have as well. You can take that information and extrapolate it into a real set of use cases, think about the size of the market of "customers who have this problem," and decide if it's something that makes sense to tackle with your product. Then you can design features to solve the real problem. If you just have a feature factory then all you'll do is build the feature as the prospect specified. You'll go back, they'll still say no, and now you have no idea how to market this feature or if it would even be useful for anyone else or it's just built around the customer's internal process.
- goopthink 4y ago
  OP here -- something I didn't touch on is that this is really common when a company transitions from early-stage to mid stage. The operations of a small company and the operations of a mid size company (and the operations of a large company) are completely different. The problems start to happen there is a mismatch between what you are and what you think you are. If you implement big-company practices for a small startup, you're often going to be overburdening the team and losing agility. When you are a med/large company but operating like a small company, you'll see chaos and inefficiency. Part of the problem is that there's no clear line. Observant engineering and product leaders tend to have an intuition for when practices need to change... I think team size and revenue/contract size (for B2B) are good heuristics. But more often than not, it's a stumble through the transition and folks feel the pain before figuring out the solution.
  - dcow 4y ago
    > Part of the problem is that there's no clear line. This is what baffles/wonders me about the state of the art in building companies. It seems so anecdotal and ad-hoc. Case studies and good advice, while they exist, are from "previous generations" and "we do it differently" because "culture". It feels like society has to continuously reinvent solutions to managing people and projects rather than building upon proven and effective strategies. I guess, perhaps, I am betraying my own lack of knowledge of this domain. Maybe there are tried and true tactics and I've just never experienced them. Related, it really feels like there should be a sub-field of "sociology" set on improving the efficiency of groups of humans using objective methodology that yields consistent and reliable results...
    - sumanthvepa 4y ago
      > it feels like there should be a sub-field of “sociology” set on improving the efficiency groups of humans Yes there is. Its called organisational behaviour. Most b-schools have some faculty specialising in it. Cornell,I think, has a whole department in the school of Industrial and Labor Relations. They’re pretty good.
    - danielmarkbruce 4y ago
SemanticStrengh 4y ago
hey that's called modern physics open-problems
marktangotango 4y ago
After the MVP is found, then the real work begins in building a company. We all understand that if everything is a priority, then nothing is, but often times, business resources don't. Support wants bugs fixed to keep customers happy and to re-sign. Sales whats features to land their whales. Engineering wants to work off tech debt, refine, and innovate. Etc.. I've found that a formal "Intake" process works wonders. Engineering needs an advocate to take these disparate priorities and refine what, when and how to address. It's really that simple, and that complex. Finding someone to own it. Sometimes it's impossible depending on the Org and personalities involved.
- jrvarela56 4y ago
  Think you nailed it by breaking it up by groups of people with different lists. Literally every group/dept/team has a list. Someone/process has to arrange that into a single list for the group that builds. If you have more than one group that builds stuff, then more output lists and more complex the transformation from input lists to output lists. Guess this is a simplistic summary of what PMs and Eng managers do.
- hinkley 4y ago
  > Engineering wants to work off tech debt, refine, and innovate. I know I'm not the only one who is backing away slowly from the concept of tech debt, and I'm not the only one trying to drag other people with me. Tech debt has turned into hippie-dippie bullshit in the minds of many people. It was an attempt at enlightened self interest by appealing to the bean counters, but if you're talking to bean counters you've already lost. The developers want easy things to be easy so they don't have to spend half their time slogging through code that does everything the wrong way, half their time explaining why everything takes so long (heading toward infinity), and half their time implementing new functionality. And yes that's 3/2 which is why we have people doing 60 hour work weeks. A long time ago I saw a speaker claim that it's not tech debt, it's wear and tear. I don't know if I agree entirely with that characterization, but I definitely think that it's less wrong.
  - pgwhalen 4y ago
perlgeek 4y ago
> Be really careful with the yellows: if there is low impact, you might be confusing work for progress. If there is low urgency, you might be overestimaging the value of your work. On the flipside, if only high-urgency projects get done, you end up becoming a 500+ employee IT service provider without a proper Identity and Access Management System (IAM), without a Privileged Access Management system (PAM), a haphazard, ad-hoc secrets store and so on. That's all fine and good for a 10-person company, you can get by like that when you're 100, above 200 maybe it starts to get icky... Yes, I understand that if really nothing progresses, you have to prioritize ruthlessly, but you cannot keep doing only high urgency projects forever.
- jbjbjbjb 4y ago
  I guess those things would eventually become urgent.
  - dhzhzjsbevs 4y ago
    Pretty much. Not sure why you'd want to define these processes so early. Don't even know what you need / what will / won't scale.
  - xbryanx 4y ago
    Yes, but there are things that are high risk enough that they must be addressed before they are urgent.
ketzo 4y ago
No real comment other than damn, what a solid-gold piece of writing and advice. Felt like I was just nodding along to every other sentence. Awesome stuff. I particularly liked breaking down “priority” into the intersection of urgency and impact.
- metalrain 4y ago
  I wonder how much that helps, obviously you can try to assign some expected value or maybe few different scenarios on how value might realize, but then you have two different knobs to play with. If your organization has lots of politics you might be willing to "adjust" urgency of your projects by setting tighter than normal timelines or boost projected value such that your project gets chosen. More simple low-mid-hi model doesn't give that detail so people understand that it's just a guess, not exact calculation.
  - BarryMilo 4y ago
    If nothing else, the psychological difference is invaluable. "Priority" is irrational and often arbitrary. Impact means money. Urgency means timeline. These things can be put in numbers, even if inaccurate, this is why it's much easier to build consensus around them. Regarding politics, the author mentioned aligning incentives in another comment. It seems generally necessary regardless of the method used.
mandeepj 4y ago
To those who are interviewing these days for a leadership position, the article has details for answering questions related to managing conflicting\multiple priorities
eastbayjake 4y ago
This was a really great article and case study. The problems (and eventual solutions) will be familiar and unsurprising to readers of The Phoenix Project: https://itrevolution.com/the-phoenix-project/ https://itrevolution.com/the-phoenix-project/
seydor 4y ago
When Everything is Grey but Nothing is Getting Read
opportune 4y ago
A lot of this is, IMO, very basic - a project is not done until it’s tested and running in prod - I am surprised anywhere with engineers with any experience at all would act otherwise. I don’t think having 40 people work on one thing at once is good at all. It’s like solving scaling by just getting a faster machine. Eventually you may not be able to purchase a machine fast enough to do everything you need to do… And you’re going to have a ton of deadweight. I hate “agile” as practiced but, in theory, it solves this. If you scope out a project well, you should have a general sense for what cross-team dependencies exist and be able to put it on the other team’s plate. You need a decent amount of “trust” but ideally the other team will actually appropriately prioritize it, ie if it takes only two days of work to unblock a project with big business impact, they won’t push back that their Epic Rewrite v3 is too important and maybe they’ll get to it in 6 months. The biggest problem I’ve seen is with bucketing all projects in long planning cycles (3 months, 6 months, 12 months, etc.). Every cross team dependency then takes about one planning cycle to be accomplished, and unexpected additional work/dependencies become nightmares. Also, while “ownership” and specialization are good to an extent, fixing a particular engineer to a specific domain over long intervals almost always causes problems because X% of engineers either aren’t skilled enough to be able to handle owning a domain, leave the team, get delayed, etc. If everybody is fully booked for a long time, there isn’t slack to help them, and it suddenly becomes a Big Deal to assign someone off a domain (because the person struggles for months and messes up a big project, rather than identifying they are messing up after weeks and course-correcting by putting them on something they can be more productive working on), and every project with more than a few people gets delayed because there’s no way to fix one or two engineers on the critical path dropping the ball. What both this article and agile touch on is the benefit of having a flexible planning structure and not over-committing or trying to plan too far in advance. IMO, that is the key to getting shit done. I think you can do that without taking it to the extreme of having only one project in-flight though.
davesque 4y ago
There are some other dynamics that are worth mentioning here that can stand in the way of undertaking any of the steps the author recommends. I don't think they touched on any of these things so I'll list some of them out: * Technical staff in an organization feel they have no authority or respect within the organization. This can happen when companies become too sales-driven. * Senior technical staff feel they have no ability to delegate, either because of the previous point or because they don't trust the technical ability of less senior staff. * Those in an organization who do have authority to delegate lack sufficient understanding of priorities or lack technical knowledge. These things highlight the classic balance between technical knowledge and managerial skills. If you don't have people that bring both to the table, you're in for trouble. Not sure if I'd go so far as to say this is what it's all about. However, reading over the author's advice, I thought to myself, "None of this could have happened at some places I've worked." You can come up with a good plan, but if no one will listen to you or take your advice, it's worthless. The author probably was that person who brought what was needed to the table. As such, perhaps the plan is not what ended up mattering so much as having someone who was willing and able to push it through. Also, having the right people from top to bottom. My feeling sometimes is that effective teams occur almost randomly. This might be because organizations rarely have the patience or the resources to assemble them deliberately.
- goopthink 4y ago
  OP here. I think those are good points. Particularly: "perhaps the plan is not what ended up mattering so much as having someone who was willing and able to push it through.". This is very true. It was a combination of having tenacity to grind through a lot of politics and repetitive conversations and doubts that everyone had, while also addressing extremely real technical and process issues. Previous leaders often brought one or the other to the team: they either knew what needed to be done technically but couldn't push a plan through, or were politically savvy enough to get changes implemented but they just ended up moving deckchairs around. When I stepped in, I inherited a team that did this twice previously and everyone had zero trust and learned helplessness. I wrote this up and tried to emphasize the slowness of the process and the fact that it wasn't a "one simple trick" type approach because that was the case. Or: "If no one will listen to you or take your advice, it's worthless." This is true. We had a few really brilliant developers cursed with a cassandra complex: they correctly identified the problems but given their way of communicating, they actually fostered resistance to the changes. Communication of the plan and building consensus is as important as the plan itself, if not moreso. Every plan will need to be adjusted based on reality and context, but as long as you build consensus and goodwill, you'll be able to have that flexibility. With regards to your point about technical staff, we were an enterprise sales organization where decision making was concentrated in the finance and sales departments. It was extremely sales driven. The reason we were able to push this through was precisely because the company had lost faith in product and engineering to accomplish anything after a series of poor launches and chronically missed deadlines. I actually think the problem is that if engineering leadership did have more trust, they would have pushed back on this. It didn't look good to "give up and try to just do one thing at a time" as we did. For most organizations, that's painfully radical. But in the words of Munger/Buffet: "Don't just do something, sit there!"
dhzhzjsbevs 4y ago
I don't get it. How did you get to 300 projects with only 40 engineers? Sounds like you have no strategy at all.
- danielmarkbruce 4y ago
  It's company wide - many things which aren't the product still require or affect engineerinig. I'd bet that almost every company with 40 engineers has 300+ projects. Every company I ever worked at tried to do waaaayyyyy too much at once. It might be the single biggest problem in business. Steve Jobs is lauded for his design skills, but I'd guess it was the focus thing which made the difference. Warren Buffett similar. Bill Gates too. They have all commented on it, and it rings true to anyone who has spent time in a few businesses.
  - dhzhzjsbevs 4y ago
    The only context to that 30 number given was the 40 engineers. It's highly likely they're all engineering projects. But to be fair, even if they have a 10:1 ratio of non devs that's still what 300 projects across < 500 people? That's way too much. I get pissy at product when they try to give us more than one goal per sprint. If someone tried to give us multiple projects I'd laugh at them and ask them why they're cancelling the active one.
    - danielmarkbruce 4y ago
      You don't have a backlog? He was listing every project the company was planning to do. If you go through your backlog you'll find over 100, without fail. And, yes, it's way too much. Which was the entire point.
devnull3 4y ago
As someone who have gone through this there is one Achilles heel: appraisals. Unless the management agrees in letter & spirit on prioritisation this is a disaster. When everything is a priority the message from top is "We want best of both worlds" i.e. we want productivity, quality and less complaining. Now you have prioritized in seq: A B C D... The team who has C or D to deliver will have there deliverables pushed by say 6 months. Guess who will be pissed-off with this? But the team accountable for D will say that this is what was agreed and signed on but the top still had those flawed expectations back of their mind. There is also cross-team impact. The product manager who was championing project D now has his/her appraisal in question. He/She will point fingers to the engineering and complain to his/her boss. In other words: Every one in the chain of command has vested interest. Unless they are aligned to large extent its hard to get out of this situation.
- goopthink 4y ago
  This is a really valuable point and we faced a lot of ground-level resistance because of this. Teams and people were incentivized to care about the wrong things. Those things made sense in the past, but the structure and incentives of the engineering org did not keep up with the changing business needs. Again: not a knock on historic decisions (which made sense at the time), but a reflect of necessary change. Changing the incentive structure was implicit in the reorg we did and it was one of the most painful parts of the process. That said, the reorg allowed for more effective parallelization by making teams more independent, and thereby avoiding the "I'm sitting here and unable to get anything done" problem. But yes, if people are not in consensus, you'll have sabotage. The Lippit-Knoster model for managing complex change does a great job highlighting this.
- xcambar 4y ago
  The author seems well aware of that and wrote than all of this is only truly actionable if/when you're at a director-like level, and you can handle expectations from below and above.
lucidguppy 4y ago
Its not tech debt. Its rigidity and fragility. If you only add features, you gain rigidity and fragility. To go fast you must go well. Teams that balance maintainability against features have a competitive advantage and will beat you every time. Going "fast" immediately slows you down. Right then and there. You think you're going fast but you're just fooling yourself. The DevOps Handbook goes into this - you need to spend 20% of the teams time on clean code and maintainability.
- xwolfi 4y ago
  Not 19, not 21, 20. That's how you remove rigidity? Most places who apply handbooks end up applying handbooks to the detriments of everything else. I feel we need to embrace the fact software is a fluid monster and we must work at it dynamically: cleaning up when it s unbearably crappy, adding to it when clients and users depend on it, and a balance between both when clients can wait a bit or when the dirt hasn't settled yet. Then you d have a team that delivers rather than state that we must lose the client because his need overlap our 20% "moving classes around more logical namespaces" clean code.
  - mike_hock 4y ago
    > Not 19, not 21, 20. That's how you remove rigidity? I mean, the fact that the 20% figure is only a rough guideline is so obvious it doesn't need to be stated.
  - Cpoll 4y ago
    > Then you d have a team that delivers rather than state that we must lose the client because his need overlap our 20% "moving classes around more logical namespaces" clean code. Which sounds like something that happened zero times ever. The problem is rarely that there's too much tech debt work, it's that client requirements take over dev work right up until the product goes up in flames (at which point the client you meticulously built features for disappears along with the rest). Since you're rejecting handbooks, how do you define "unbearably crappy?" Perhaps it's when the devs have already started looking for new jobs? And then, if it's unbearably crappy and you start cleaning up and a client comes with requirements, do you drop the cleanup work?
dmtroyer 4y ago
“… a learned helplessness where the only solution seemed to be resignation.” oof. too real.
dclowd9901 4y ago
> Figuring out the impact and likelihood was something that a product management org would normally devote the bulk of its time towards. Here’s a question: why weren’t they devoting the bulk of their time to it? I’ve been in sessions like this as an engineer and they end up being incredibly productive and informative for everyone as to what the priorities actually are. These aren’t things that engineers are supposed to be doing better than product managers. Why do product orgs _constantly_ fall on their faces when it comes to prioritization? My guess is that the job is highly politics centric and PMs have a hard time getting out of the way of their own aspirations or ego to let the progress take precedence. And no this isn’t trite PM bashing. I would like organizations to re-evaluate the incentive structures of product owners and make them look more like tech stack performance metrics (literally, as in how much is our software getting out of the way of our customers usage and allowing them to be “productive”, whatever that term may mean for the company). Also, give product owners some credit for backing off on new features to make headway for tech debt pay down.
xnorswap 4y ago
Related to this, and something I should write up in more detail one day: Priority is an order. "High", "Medium", "Low" are not a priority, they are useless. A priority is an ordering of tasks. Once you free yourself from trying to assign labels to the priority then you free stakeholders from worrying about putting things "very high" in case they tread on toes and you enable an honest conversation about where it belongs in the list of tasks or projects that need finishing. After all, no-one ever marks things as "low" priority because they understand it'll never get done, and no-one ever marks things as "high" priority because they worry about the politics of doing so. The result is you get "medium", "Just above medium", "Just above just above medium" and "JAJAM+" in your JIRA. (True story! And a shout out to anyone recognising JAJAM+).
bdg 4y ago
Whenever I encounter issues like this: > This routinely caused the team’s focus to change to whatever was the problem of the week (or to solve for whomever was yelling loudest). My mind immediately goes to a lack of leadership and strategy. Why do we have pirates going around from team to team yelling, inventing new feature requests? A lot of what I can read here sounds like someone (goopthink) finally took ownership and seized authority from a leadership team that had rotated out twice and had mostly given up. > we had cycled through two previous heads of product and one head of engineering > In fact, I volunteered for the role because no one wanted to take on a project > The problem is that you need a conflict resolution and effort coordination mechanism > Figuring out the impact and likelihood was something that a product management org would normally devote the bulk of its time towards. Not having that information was one of the failures of process we were dealing with > adjusting the stack rank continued to be a deeply political process even after it was created > “you’re a team of 40 people… surely you can do more than one thing at a time?” > There was no way to scale those conversations. These 1:1s also allowed us to get ahead of influential senior managers who could have been saboteurs if they didn’t buy in. This whole thing screams that there is no acting leadership in the sense of building a team focused on one goal. Allowing many managers to go around and create new business lines, not holding them accountable to drive outcomes but rather just be busy with some new idea. The issue your engineers have is identical to the one going on outside of engineering: no unified mission, nobody willing to say "no" to "some new idea that could totally make us some money and we should just try it and see". It is as if the executive allowed a rabble of random managers to do whatever they wanted like gladiators in a combat arena. It's like CV driven development: "hey let's use kafka and http2 for our internal CRUD app" but instead it is like "hey let's start a new division of the business that has an entirely different workflow but exists in the same industry vertical". The leadership team is absent from every point of this story and things could only improve when they got out of the way or were engineered out with pre-messaging in 1:1s. Good that OP could improve all of this (which in itself is a huge accomplishment, something like 60% of change management efforts fail) but honestly until that L-team is rotated out entirely the problem isn"t solved, those pirates will go back on their habits that created a dysfunctional org full of "yeah but I have a cool idea" leaders. These leaders tend to want all of the decisions and none of the details, making everyone live in their mess. OP dove head-first into the details and this is why they could fix things.
test1235 4y ago
OP: typo @ "Create a clear rinish line"
- goopthink 4y ago
  Mortifying, but thanks for the catch! Should be fixed now.

389 points by goopthink 4y ago

My gut reaction to the title of the post - fire 80% of the managers. Then announce skimpy buyouts for any employee who feels that they won't get enough face time with managers and/or time spent in meetings with the new manager/worker ratio.

lbriner 4y ago

Lots of really good points in here. I have worked at various companies who have struggled after the initial growth. Hubris can take over and we feel the need to over-extend, to sit at the big-boys table, to court the corporates with their fat (but very tight) wallets. I think a common cause is simply fear. There is a lot at stake when you have 50+ employees. Maybe those one or two markets are not big enough to support what we need any more so instead of razor focus and discipline which helps, we try and build everything for everyone and a lot of it is simply wasted effort. We also start having teams so instead of a design decision being taken by the entire management (and can get shouted down), it just gets done because "the design team" regardless of the value it is actually adding or not. In extremis, we see the FANG companies who have so many engineering staff, we can only hypothesize about what most of these people even do for 40 hours per week on massive salaries.

bombcar 4y ago
I wonder how much lost productivity comes from people saying "if only it had feature X" to salesmen to be polite instead of a simple "we're not interested and won't be buying".
- goopthink 4y ago
  "The Mom Test" is an awesome look at that -- how bad questions and worse feedback leads to a lot of wasted effort. Highly recommend it. http://momtestbook.com/ http://momtestbook.com/
- mason55 4y ago
  It's the difference between being a feature factory and having a real product strategy. If you actually have some kind of customer in mind, and you have ideas of problems that you're solving for them, then this can be valuable feedback even if it's a lie. Digging into the feature might expose you to a new customer problem that weren't aware of. You can talk to other customers and prospects and understand if this is a problem they have as well. You can take that information and extrapolate it into a real set of use cases, think about the size of the market of "customers who have this problem," and decide if it's something that makes sense to tackle with your product. Then you can design features to solve the real problem. If you just have a feature factory then all you'll do is build the feature as the prospect specified. You'll go back, they'll still say no, and now you have no idea how to market this feature or if it would even be useful for anyone else or it's just built around the customer's internal process.
goopthink 4y ago
OP here -- something I didn't touch on is that this is really common when a company transitions from early-stage to mid stage. The operations of a small company and the operations of a mid size company (and the operations of a large company) are completely different. The problems start to happen there is a mismatch between what you are and what you think you are. If you implement big-company practices for a small startup, you're often going to be overburdening the team and losing agility. When you are a med/large company but operating like a small company, you'll see chaos and inefficiency. Part of the problem is that there's no clear line. Observant engineering and product leaders tend to have an intuition for when practices need to change... I think team size and revenue/contract size (for B2B) are good heuristics. But more often than not, it's a stumble through the transition and folks feel the pain before figuring out the solution.
- dcow 4y ago
  > Part of the problem is that there's no clear line. This is what baffles/wonders me about the state of the art in building companies. It seems so anecdotal and ad-hoc. Case studies and good advice, while they exist, are from "previous generations" and "we do it differently" because "culture". It feels like society has to continuously reinvent solutions to managing people and projects rather than building upon proven and effective strategies. I guess, perhaps, I am betraying my own lack of knowledge of this domain. Maybe there are tried and true tactics and I've just never experienced them. Related, it really feels like there should be a sub-field of "sociology" set on improving the efficiency of groups of humans using objective methodology that yields consistent and reliable results...
  - sumanthvepa 4y ago
    > it feels like there should be a sub-field of “sociology” set on improving the efficiency groups of humans Yes there is. Its called organisational behaviour. Most b-schools have some faculty specialising in it. Cornell,I think, has a whole department in the school of Industrial and Labor Relations. They’re pretty good.
  - danielmarkbruce 4y ago

SemanticStrengh 4y ago

hey that's called modern physics open-problems

marktangotango 4y ago

After the MVP is found, then the real work begins in building a company. We all understand that if everything is a priority, then nothing is, but often times, business resources don't. Support wants bugs fixed to keep customers happy and to re-sign. Sales whats features to land their whales. Engineering wants to work off tech debt, refine, and innovate. Etc.. I've found that a formal "Intake" process works wonders. Engineering needs an advocate to take these disparate priorities and refine what, when and how to address. It's really that simple, and that complex. Finding someone to own it. Sometimes it's impossible depending on the Org and personalities involved.

jrvarela56 4y ago
Think you nailed it by breaking it up by groups of people with different lists. Literally every group/dept/team has a list. Someone/process has to arrange that into a single list for the group that builds. If you have more than one group that builds stuff, then more output lists and more complex the transformation from input lists to output lists. Guess this is a simplistic summary of what PMs and Eng managers do.
hinkley 4y ago
> Engineering wants to work off tech debt, refine, and innovate. I know I'm not the only one who is backing away slowly from the concept of tech debt, and I'm not the only one trying to drag other people with me. Tech debt has turned into hippie-dippie bullshit in the minds of many people. It was an attempt at enlightened self interest by appealing to the bean counters, but if you're talking to bean counters you've already lost. The developers want easy things to be easy so they don't have to spend half their time slogging through code that does everything the wrong way, half their time explaining why everything takes so long (heading toward infinity), and half their time implementing new functionality. And yes that's 3/2 which is why we have people doing 60 hour work weeks. A long time ago I saw a speaker claim that it's not tech debt, it's wear and tear. I don't know if I agree entirely with that characterization, but I definitely think that it's less wrong.
- pgwhalen 4y ago

perlgeek 4y ago

> Be really careful with the yellows: if there is low impact, you might be confusing work for progress. If there is low urgency, you might be overestimaging the value of your work. On the flipside, if only high-urgency projects get done, you end up becoming a 500+ employee IT service provider without a proper Identity and Access Management System (IAM), without a Privileged Access Management system (PAM), a haphazard, ad-hoc secrets store and so on. That's all fine and good for a 10-person company, you can get by like that when you're 100, above 200 maybe it starts to get icky... Yes, I understand that if really nothing progresses, you have to prioritize ruthlessly, but you cannot keep doing only high urgency projects forever.

jbjbjbjb 4y ago
I guess those things would eventually become urgent.
- dhzhzjsbevs 4y ago
  Pretty much. Not sure why you'd want to define these processes so early. Don't even know what you need / what will / won't scale.
- xbryanx 4y ago
  Yes, but there are things that are high risk enough that they must be addressed before they are urgent.

ketzo 4y ago

No real comment other than damn, what a solid-gold piece of writing and advice. Felt like I was just nodding along to every other sentence. Awesome stuff. I particularly liked breaking down “priority” into the intersection of urgency and impact.

metalrain 4y ago
I wonder how much that helps, obviously you can try to assign some expected value or maybe few different scenarios on how value might realize, but then you have two different knobs to play with. If your organization has lots of politics you might be willing to "adjust" urgency of your projects by setting tighter than normal timelines or boost projected value such that your project gets chosen. More simple low-mid-hi model doesn't give that detail so people understand that it's just a guess, not exact calculation.
- BarryMilo 4y ago
  If nothing else, the psychological difference is invaluable. "Priority" is irrational and often arbitrary. Impact means money. Urgency means timeline. These things can be put in numbers, even if inaccurate, this is why it's much easier to build consensus around them. Regarding politics, the author mentioned aligning incentives in another comment. It seems generally necessary regardless of the method used.

mandeepj 4y ago

To those who are interviewing these days for a leadership position, the article has details for answering questions related to managing conflicting\multiple priorities

eastbayjake 4y ago

This was a really great article and case study. The problems (and eventual solutions) will be familiar and unsurprising to readers of The Phoenix Project: https://itrevolution.com/the-phoenix-project/ https://itrevolution.com/the-phoenix-project/

seydor 4y ago

When Everything is Grey but Nothing is Getting Read

opportune 4y ago

A lot of this is, IMO, very basic - a project is not done until it’s tested and running in prod - I am surprised anywhere with engineers with any experience at all would act otherwise. I don’t think having 40 people work on one thing at once is good at all. It’s like solving scaling by just getting a faster machine. Eventually you may not be able to purchase a machine fast enough to do everything you need to do… And you’re going to have a ton of deadweight. I hate “agile” as practiced but, in theory, it solves this. If you scope out a project well, you should have a general sense for what cross-team dependencies exist and be able to put it on the other team’s plate. You need a decent amount of “trust” but ideally the other team will actually appropriately prioritize it, ie if it takes only two days of work to unblock a project with big business impact, they won’t push back that their Epic Rewrite v3 is too important and maybe they’ll get to it in 6 months. The biggest problem I’ve seen is with bucketing all projects in long planning cycles (3 months, 6 months, 12 months, etc.). Every cross team dependency then takes about one planning cycle to be accomplished, and unexpected additional work/dependencies become nightmares. Also, while “ownership” and specialization are good to an extent, fixing a particular engineer to a specific domain over long intervals almost always causes problems because X% of engineers either aren’t skilled enough to be able to handle owning a domain, leave the team, get delayed, etc. If everybody is fully booked for a long time, there isn’t slack to help them, and it suddenly becomes a Big Deal to assign someone off a domain (because the person struggles for months and messes up a big project, rather than identifying they are messing up after weeks and course-correcting by putting them on something they can be more productive working on), and every project with more than a few people gets delayed because there’s no way to fix one or two engineers on the critical path dropping the ball. What both this article and agile touch on is the benefit of having a flexible planning structure and not over-committing or trying to plan too far in advance. IMO, that is the key to getting shit done. I think you can do that without taking it to the extreme of having only one project in-flight though.

davesque 4y ago

There are some other dynamics that are worth mentioning here that can stand in the way of undertaking any of the steps the author recommends. I don't think they touched on any of these things so I'll list some of them out: * Technical staff in an organization feel they have no authority or respect within the organization. This can happen when companies become too sales-driven. * Senior technical staff feel they have no ability to delegate, either because of the previous point or because they don't trust the technical ability of less senior staff. * Those in an organization who do have authority to delegate lack sufficient understanding of priorities or lack technical knowledge. These things highlight the classic balance between technical knowledge and managerial skills. If you don't have people that bring both to the table, you're in for trouble. Not sure if I'd go so far as to say this is what it's all about. However, reading over the author's advice, I thought to myself, "None of this could have happened at some places I've worked." You can come up with a good plan, but if no one will listen to you or take your advice, it's worthless. The author probably was that person who brought what was needed to the table. As such, perhaps the plan is not what ended up mattering so much as having someone who was willing and able to push it through. Also, having the right people from top to bottom. My feeling sometimes is that effective teams occur almost randomly. This might be because organizations rarely have the patience or the resources to assemble them deliberately.

goopthink 4y ago
OP here. I think those are good points. Particularly: "perhaps the plan is not what ended up mattering so much as having someone who was willing and able to push it through.". This is very true. It was a combination of having tenacity to grind through a lot of politics and repetitive conversations and doubts that everyone had, while also addressing extremely real technical and process issues. Previous leaders often brought one or the other to the team: they either knew what needed to be done technically but couldn't push a plan through, or were politically savvy enough to get changes implemented but they just ended up moving deckchairs around. When I stepped in, I inherited a team that did this twice previously and everyone had zero trust and learned helplessness. I wrote this up and tried to emphasize the slowness of the process and the fact that it wasn't a "one simple trick" type approach because that was the case. Or: "If no one will listen to you or take your advice, it's worthless." This is true. We had a few really brilliant developers cursed with a cassandra complex: they correctly identified the problems but given their way of communicating, they actually fostered resistance to the changes. Communication of the plan and building consensus is as important as the plan itself, if not moreso. Every plan will need to be adjusted based on reality and context, but as long as you build consensus and goodwill, you'll be able to have that flexibility. With regards to your point about technical staff, we were an enterprise sales organization where decision making was concentrated in the finance and sales departments. It was extremely sales driven. The reason we were able to push this through was precisely because the company had lost faith in product and engineering to accomplish anything after a series of poor launches and chronically missed deadlines. I actually think the problem is that if engineering leadership did have more trust, they would have pushed back on this. It didn't look good to "give up and try to just do one thing at a time" as we did. For most organizations, that's painfully radical. But in the words of Munger/Buffet: "Don't just do something, sit there!"

dhzhzjsbevs 4y ago

I don't get it. How did you get to 300 projects with only 40 engineers? Sounds like you have no strategy at all.

danielmarkbruce 4y ago
It's company wide - many things which aren't the product still require or affect engineerinig. I'd bet that almost every company with 40 engineers has 300+ projects. Every company I ever worked at tried to do waaaayyyyy too much at once. It might be the single biggest problem in business. Steve Jobs is lauded for his design skills, but I'd guess it was the focus thing which made the difference. Warren Buffett similar. Bill Gates too. They have all commented on it, and it rings true to anyone who has spent time in a few businesses.
- dhzhzjsbevs 4y ago
  The only context to that 30 number given was the 40 engineers. It's highly likely they're all engineering projects. But to be fair, even if they have a 10:1 ratio of non devs that's still what 300 projects across < 500 people? That's way too much. I get pissy at product when they try to give us more than one goal per sprint. If someone tried to give us multiple projects I'd laugh at them and ask them why they're cancelling the active one.
  - danielmarkbruce 4y ago
    You don't have a backlog? He was listing every project the company was planning to do. If you go through your backlog you'll find over 100, without fail. And, yes, it's way too much. Which was the entire point.

devnull3 4y ago

As someone who have gone through this there is one Achilles heel: appraisals. Unless the management agrees in letter & spirit on prioritisation this is a disaster. When everything is a priority the message from top is "We want best of both worlds" i.e. we want productivity, quality and less complaining. Now you have prioritized in seq: A B C D... The team who has C or D to deliver will have there deliverables pushed by say 6 months. Guess who will be pissed-off with this? But the team accountable for D will say that this is what was agreed and signed on but the top still had those flawed expectations back of their mind. There is also cross-team impact. The product manager who was championing project D now has his/her appraisal in question. He/She will point fingers to the engineering and complain to his/her boss. In other words: Every one in the chain of command has vested interest. Unless they are aligned to large extent its hard to get out of this situation.

goopthink 4y ago
This is a really valuable point and we faced a lot of ground-level resistance because of this. Teams and people were incentivized to care about the wrong things. Those things made sense in the past, but the structure and incentives of the engineering org did not keep up with the changing business needs. Again: not a knock on historic decisions (which made sense at the time), but a reflect of necessary change. Changing the incentive structure was implicit in the reorg we did and it was one of the most painful parts of the process. That said, the reorg allowed for more effective parallelization by making teams more independent, and thereby avoiding the "I'm sitting here and unable to get anything done" problem. But yes, if people are not in consensus, you'll have sabotage. The Lippit-Knoster model for managing complex change does a great job highlighting this.
xcambar 4y ago
The author seems well aware of that and wrote than all of this is only truly actionable if/when you're at a director-like level, and you can handle expectations from below and above.

lucidguppy 4y ago

Its not tech debt. Its rigidity and fragility. If you only add features, you gain rigidity and fragility. To go fast you must go well. Teams that balance maintainability against features have a competitive advantage and will beat you every time. Going "fast" immediately slows you down. Right then and there. You think you're going fast but you're just fooling yourself. The DevOps Handbook goes into this - you need to spend 20% of the teams time on clean code and maintainability.

xwolfi 4y ago
Not 19, not 21, 20. That's how you remove rigidity? Most places who apply handbooks end up applying handbooks to the detriments of everything else. I feel we need to embrace the fact software is a fluid monster and we must work at it dynamically: cleaning up when it s unbearably crappy, adding to it when clients and users depend on it, and a balance between both when clients can wait a bit or when the dirt hasn't settled yet. Then you d have a team that delivers rather than state that we must lose the client because his need overlap our 20% "moving classes around more logical namespaces" clean code.
- mike_hock 4y ago
  > Not 19, not 21, 20. That's how you remove rigidity? I mean, the fact that the 20% figure is only a rough guideline is so obvious it doesn't need to be stated.
- Cpoll 4y ago
  > Then you d have a team that delivers rather than state that we must lose the client because his need overlap our 20% "moving classes around more logical namespaces" clean code. Which sounds like something that happened zero times ever. The problem is rarely that there's too much tech debt work, it's that client requirements take over dev work right up until the product goes up in flames (at which point the client you meticulously built features for disappears along with the rest). Since you're rejecting handbooks, how do you define "unbearably crappy?" Perhaps it's when the devs have already started looking for new jobs? And then, if it's unbearably crappy and you start cleaning up and a client comes with requirements, do you drop the cleanup work?

dmtroyer 4y ago

“… a learned helplessness where the only solution seemed to be resignation.” oof. too real.

dclowd9901 4y ago

> Figuring out the impact and likelihood was something that a product management org would normally devote the bulk of its time towards. Here’s a question: why weren’t they devoting the bulk of their time to it? I’ve been in sessions like this as an engineer and they end up being incredibly productive and informative for everyone as to what the priorities actually are. These aren’t things that engineers are supposed to be doing better than product managers. Why do product orgs _constantly_ fall on their faces when it comes to prioritization? My guess is that the job is highly politics centric and PMs have a hard time getting out of the way of their own aspirations or ego to let the progress take precedence. And no this isn’t trite PM bashing. I would like organizations to re-evaluate the incentive structures of product owners and make them look more like tech stack performance metrics (literally, as in how much is our software getting out of the way of our customers usage and allowing them to be “productive”, whatever that term may mean for the company). Also, give product owners some credit for backing off on new features to make headway for tech debt pay down.

xnorswap 4y ago

Related to this, and something I should write up in more detail one day: Priority is an order. "High", "Medium", "Low" are not a priority, they are useless. A priority is an ordering of tasks. Once you free yourself from trying to assign labels to the priority then you free stakeholders from worrying about putting things "very high" in case they tread on toes and you enable an honest conversation about where it belongs in the list of tasks or projects that need finishing. After all, no-one ever marks things as "low" priority because they understand it'll never get done, and no-one ever marks things as "high" priority because they worry about the politics of doing so. The result is you get "medium", "Just above medium", "Just above just above medium" and "JAJAM+" in your JIRA. (True story! And a shout out to anyone recognising JAJAM+).

bdg 4y ago

Whenever I encounter issues like this: > This routinely caused the team’s focus to change to whatever was the problem of the week (or to solve for whomever was yelling loudest). My mind immediately goes to a lack of leadership and strategy. Why do we have pirates going around from team to team yelling, inventing new feature requests? A lot of what I can read here sounds like someone (goopthink) finally took ownership and seized authority from a leadership team that had rotated out twice and had mostly given up. > we had cycled through two previous heads of product and one head of engineering > In fact, I volunteered for the role because no one wanted to take on a project > The problem is that you need a conflict resolution and effort coordination mechanism > Figuring out the impact and likelihood was something that a product management org would normally devote the bulk of its time towards. Not having that information was one of the failures of process we were dealing with > adjusting the stack rank continued to be a deeply political process even after it was created > “you’re a team of 40 people… surely you can do more than one thing at a time?” > There was no way to scale those conversations. These 1:1s also allowed us to get ahead of influential senior managers who could have been saboteurs if they didn’t buy in. This whole thing screams that there is no acting leadership in the sense of building a team focused on one goal. Allowing many managers to go around and create new business lines, not holding them accountable to drive outcomes but rather just be busy with some new idea. The issue your engineers have is identical to the one going on outside of engineering: no unified mission, nobody willing to say "no" to "some new idea that could totally make us some money and we should just try it and see". It is as if the executive allowed a rabble of random managers to do whatever they wanted like gladiators in a combat arena. It's like CV driven development: "hey let's use kafka and http2 for our internal CRUD app" but instead it is like "hey let's start a new division of the business that has an entirely different workflow but exists in the same industry vertical". The leadership team is absent from every point of this story and things could only improve when they got out of the way or were engineered out with pre-messaging in 1:1s. Good that OP could improve all of this (which in itself is a huge accomplishment, something like 60% of change management efforts fail) but honestly until that L-team is rotated out entirely the problem isn"t solved, those pirates will go back on their habits that created a dysfunctional org full of "yeah but I have a cool idea" leaders. These leaders tend to want all of the decisions and none of the details, making everyone live in their mess. OP dove head-first into the details and this is why they could fix things.

test1235 4y ago

OP: typo @ "Create a clear rinish line"

goopthink 4y ago
Mortifying, but thanks for the catch! Should be fixed now.