14 ms·
The whole technical debt discussion often makes me wonder if there's a fundamental distrust (or even disrespect) of software engineers as domain experts. Do pe
by Fr0styMatt88 7y ago
The whole technical debt discussion often makes me wonder if there's a fundamental distrust (or even disrespect) of software engineers as domain experts.
Do people in other professions have the same issues? If the salespeople say this thing must be done in this particular way, or if the lawyers say this thing must be done in this particular way, are they questioned and overridden by their managers? I honestly don't know; it feels like software engineers are given a special level of distrust but it just might be that I'm perceiving that because I haven't experienced the other jobs.
FWIW where I currently work, we have an amazing middle-manager / product owner who has to fight the same battles with upper management with regards to process improvement and technical debt. She gets it, but has to fight the same battles we do. Sales and C-level want everything now with as little investment as possible.... I wonder if I just answered my own question?
- bsaul 7y agoFrom my experience, managers tend to start to distrusts software engineers after the first complete system rewrite, that was supposed to solve the big problems, actually didn’t solve them and brought new ones instead.
- arcturus17 7y agoMaybe that’s one more reason why, paraphrasing Spolsky, you don’t do big system rewrites and instead carefully refactor the system bit by bit?
- mathattack 7y agoThe issue is neither has crossed the bridge. The non-technical manager can’t understand the risks, and the engineers can’t quantity it in the dollars and cents that CFOs understand. Salespeople are good at dollars and cents. So are some lawyers. But they get overruled all the time by execs. “I don’t care how you do it, increase sales by 20% in Q4” creates the sales and marketing version of tech debt. (Channel stuffing, excess discounting, etc)
- mobjack 7y agoI distrust my fellow software engineers when they say they need time to pay tech debt. There were many times where I refactored code that never received any additional updates or to have the feature removed. I wasted a lot of time without any real gain for it. There are real costs to paying off tech debt and it is easy for engineers to spend their time on low value refactoring. Software engineers are not always good at prioritizing the business value side of their work so they aren't trusted there. If you can show you can prioritize business value then it is easier to convince management to give you two weeks to clean up tech debt.
- missosoup 7y ago> If you can show you can prioritize business value then it is easier to convince management If your management needs 'convincing' then there's a problem. Either the manager is technical enough to know whether a particular set of work is the right thing to do or not, in which case they don't need convincing. Or they're not, in which case they should defer to the engineer and also not need convincing. The only managers who need 'convincing' on technical matters are technically inept individuals who nonetheless attempt to micromanage technical decision making. They are no more able to link business value to technical decisions than the engineers you say you distrust. Fire them.
- killbot5000 7y ago> Engineers don't generally enjoy paying off technical debt, it's a chore. But they recognise that it must be done sometimes. I disagree. Most engineers I work with like refactoring. It’s usually well defined work with clear metrics for success. It can also reduce daily pain if it makes delicate/messy code much easier to work with. The problem with refactoring is that its only value is in improved execution of future projects. So it can be extremely valuable or completely useless. It depends on how likely the refactored things are to be worked on again.
- rumanator 7y ago> Software engineers are not always good at prioritizing the business value side of their work so they aren't trusted there. This. Software engineers are focused on the implementation side of things and how today's work will impact their work in the future. Yet, sometimes a quick and dirty solution that takes close to nothing to implement is more than enough to put a functionality in production. Sure, the whole thing will need to be scrapped in the near future when rewriting the feature to meet all design requirements, naturally the quick and dirty solution underperforms, and of course implementing it properly will take 3 or 5x the time. Isn't it obvious that the team is wasting it's time by implementing a quick and dirty solution? Yet, the quick and dirty feature does shorten the time to market to a fraction of what it would otherwise take to implement properly, and perhaps that is good enough for the project for the time being.
- overgard 7y ago> The whole technical debt discussion often makes me wonder if there's a fundamental distrust (or even disrespect) of software engineers as domain experts. Yeah. I think, sadly, this is mostly on software developers though. Look at all the pop culture stereotypes of programmers: whether those stereotypes are true or not, that is what culture thinks of us. Would you take seriously what a big-bang theory character would say? The problem is, I don't see most software engineers valuing social skills to better relate to other professions. Instead, there's a lot of whining about being misunderstood. And we are misunderstood! But man we're not helping it at all.
- pm90 7y agoContrary to most stereotypes, most adult software engineers are fairly well functioning adults. They may not be the best at public speaking or other social skills, but they're not mostly like characters from the Big Bang Theory.
- overgard 7y agoI agree, but if the split is 90% well functioning, and 10% people with the manners of RMS/Torvalds, well, people are going to notice those 10%. Stereotypes obviously don't need to be real to shape things. My concern more is that our industry needs to stick up for itself and act like professionals, instead of intentionally marginalizing itself and then acting indignant about it. I don't even think that 10% really needs to change. We need the RMS' and Torvalds of the world. But we also need to, individually, act better than they do and advocate for ourselves as adults. I constantly see coders deriding any decision that places the business over their own individual interests, and then we get indignant that the decision makers don't listen to us. But if we don't care about their interests, and the collective best interest of the company, why should they care about our interests?
- acjohnson55 7y ago> Do people in other professions have the same issues? There are very few professions that have as much leeway as we do to begin with.
- Aeolun 7y agoI think this is related to how badly things go wrong if there is a fuckup. You really do not want to push the engineers responsible for determining your $500M office building will stay standing.
- Fr0styMatt88 7y agoSo what kind of fuckup in software is going to have to happen before this is also true? My fear is that it will happen one day, if not already (see eg- 737 MAX). I'm not looking forward to that day.
- pritambaral 7y agoThe 737 Max was not a software fuckup. The concerns raised by the engineers were overruled by management, and then buried during certification.
- taf2 7y agoIt’s because there are literally infinite ways to solve similar problems in software. There is no physical assets to measure the differences making very hard as relatively speaking it’s still a very young domain. I still go back to my freshmen year in college when in my dorm we had a mechanic, chemical and material science engineers. Each of them complained because they were not senior enough they could not get time in the expensive lab to run their end of year projects... it was at that moment they I looked at my crappy dell laptop and realized I had access to the most advanced systems in my field computer science and it was just sitting here in my dorm... I mean a few years later a guy on similar crappy laptop or desktop created Facebook... and yet there are so many options to build Facebook using php, c#, c++, Java, python etc... maybe to you one of those ways is inferior or less good then the other. Maybe using cloud servers or physical would be better?? Maybe a class or a function... css or Scss... the space is infinite
- lr4444lr 7y agoSpeaking collectively, given how much we fight against standardization of our profession, we should be distrusted, IMHO.
- ben_jones 7y agoWe distribute senior titles to people with three years of experience. We task junior developers with setting up and managing critical cloud infrastructure. Management has no idea how it all works. It's kind of like we're asking for it.
- fpig 7y agoAnd we often hire people who have no clue how things work even for low-level management positions. Can you imagine in construction, the lowest level manager being someone with no clue about construction but is just supposedly really good at "project management"? No way that would ever happen. The "management has no idea how it all works" issue is real but it's not because the CEO doesn't know how it works. The CEO doesn't know how things work in any industry. In SD we have people with 3 people under them who are clueless. There is no construction company in the world where a guy who has 3 people under him doesn't understand the actual work being done. No wonder there is a trust issue when there are such huge knowledge gaps, it's not uncommon that someone's first direct superior already understands very little about what they do. Although I have a feeling that this issue is not present at "top" companies. I have a suspicion that at a company like Microsoft, you'd have to go quite a few levels up the chain before you get to someone who is technically clueless. I doubt they'd hire some guy who never wrote a line of code and have him lead the Visual Studio team or whatever. This is something shitty companies do, it's not an unavoidable fact of life. Basically there should never be a situation where someone's job is to write code, but their boss can't understand that code. The knowledge gap between boss and employee is too large. A boss always needs to be able to understand the work their employee is doing. If you're at a higher management level where your direct reports do not write code, then sure, you don't need to understand code.
- Bukhmanizer 7y agoI think it’s about perception. Broadly speaking, professions like Doctors and Lawyers are seen as prestigious and important, and so they are allowed to manage themselves. I think software engineers tend to overestimate how prestigious and valuable people outside of tech think software engineers are. I work in computational biology, and even in a fairly tech related field, you wouldn’t believe how many people think software engineering is just glorified IT. I’ve had a lot of conversations where the crux of it is “why should we pay them so much money, it’s the computers that are doing the work. Or, on building a distributed computation engine to run for 2 years straight without failing: “it’s all just grunt work we’ll just get an undergrad to do it”.
- tetraca 7y ago> why should we pay them so much money, it’s the computers that are doing the work. Why should we pay the blacksmith so much money? It's the armor doing all the work. Why should we pay the stone mason so much money? It's the arch that's doing all the work.
- Bukhmanizer 7y agoYes that’s essentially the problem. Engineers/developers aren’t seen as fundamentally making anything. They’re just seen as the interface to the computer.
- tetraca 7y agoIf you cheap out on your smith, good luck coming back without a hole in you.
- milesskorpen 7y agoDoctors operate with enormous amounts of regulation and overhead; I don't think they're allowed to really manage themselves as all.
- monocasa 7y ago
- rpier001 7y agoPeople in other professions have similar issues. Some 'smart' folks feel as if they should be able to understand everything. It probably often works for them, after all, how did they become 'smart'? But, it's a fine line between genuine interest and distrust per se. Try to assume good intent, consider it interest and grow your ability to explain. At some point you might have to call it as a poor use of time and move on, but if you can use the time to grow on both sides of the discussion - bonus.
- Fr0styMatt88 7y agoThis is what I suspected, with my perception being tainted by the "grass is always greener". I also can't say we aren't collectively to blame, or at least somewhat - as a bunch we can be poor at political negotiation/skill, lack of standardization, etc. I kick myself every time I miss an estimate that I've given, because I know at some level that has to erode trust. Or when my manager does press for more details as to why something should be done and I can't come up with a really strong justification at that moment, etc.
- killjoywashere 7y ago> if there's a fundamental distrust (or even disrespect) of software engineers as domain experts. Physicians enjoy a similar problem. Everyone goes to Dr. Google an then brings their misunderstandings to the clinic. Anything goes wrong, it couldn't possibly have been the fact that the patient was a brittle diabetic with hypertension. It must be the care they received in the hospital after they had a heart attack while driving and ran into a tree. Nor is anyone else in the hospital possibly making mistakes. Clearly it's the board-certified physician who doesn't know what they're doing.
- Fr0styMatt88 7y agoOff-topic, but the ‘Dr Google’ thing is interesting. Something that makes it more nuanced in that particular case is that a patient will have better insight into the state of their own body in many ways than a physician will. I’ve been watching a series called ‘Mystery Diagnosis’ about people with rare diseases and how they were eventually diagnosed. Something you hear most of them say is along the lines of ‘Finally I met this doctor that actually listened to everything I had to say’. It’s a really interesting series — and damn does it remind you how lucky you are to be mostly healthy!
- fouc 7y agoIn a sense, technical debt is only a big problem for large monolithic codebases. Software that is primarily composed of small manageable components means that technical debt doesn't matter as much. When each component is roughly one week's worth of code, it becomes easy to delete a component with a model that no longer fits the problem domain, and rewrite a new component with a new more appropriate solution. "If you can rewrite a piece of code in a week, you can understand it in a week." - Greg Young https://maximilianhoffmann.com/talks/programming/the-art-of-destroying-software https://maximilianhoffmann.com/talks/programming/the-art-of-...
- mikekchar 7y agoExcept when you large monolithic codebase is composed of a whole bunch of screwy distributed components that have been thrown together haphazardly. And, yes, I know you can't have a "monolithic codebase" with "distributed components". It's just that it's actually all the same dish with a different sauce. Distribution of processing doesn't create better code bases -- it's decoupling of the computation that does it. You can decouple code and a very large number of ways. It's also just as easy to write a rat's nest of confusing spaghetti with components that hold each other's state as it is to do it without "components" (I speak from experience on this front, as much as I hate to admit it).
- fouc 7y agoWell if components are holding each others states then that's failing the decoupling test. But I agree, it's definitely not easy to keep things properly decoupled. Requires some level of discipline or some savvy choices up front in the overall architecture of the whole app..
- TheCoelacanth 7y agoNon-decoupled components = tech-debt
- gamerDude 7y agoI think it's a bit different in software because it's fairly easy to change things and update them. As for the distrust, generally there is can be a lot of distrust in things you don't fully understand. Legal I think is a great example. There are a lot of contracts that screw people over and legalese is so hard to read, I have a hard time trusting what is going on. But one big difference is it is very difficult to go back and change legal terms. I was changing an LLC to a c-corp in order to receive investment and it took 3 months to fix the "legal debt" I had incurred by starting with an LLC with outside parties who had a say in what the new terms were. And it's hard to say whether we did all the right moves in that transition because I'm not a legal expert. Just as some people doubt the decisions I make in software.
- mikekchar 7y agoI think one of the problems is that code is a black box to non-programmers. If I tell you that your car needs $2000 worth of work in order to avoid a $10K problem down the line, how do you evaluate that? Keep in mind that as far as you can tell the car is performing exactly as you expect at the moment. Another problem is that especially in large organizations there are people who are literally trying to scam their way into money. You always have people aggrandizing their work, or belittling others. Especially as companies become profitable, they attract people who are only out for their cut. There are people who do nothing but attend meetings and try to scoop money into their pockets. This is the norm. Now marry the two issues and you can see the result: someone says they have to do "special work" that will delay the project and that they are actually saving everybody else's asses. Nobody that isn't involved in doing the work can evaluate the claim. Are they just avoiding work/blame? How do you know? I find that usually in organisations I work with, there is trust up to the D-level manager (i.e. those who directly manager developers), but there is often very little trust from other parts of the organisation (sales, upper management, or sometimes even program managers). The best way to engender trust is to work as tranparently as possible. In the best of worlds you would deliver something every day and it would be obvious what's in the queue. Things would be in the queue for a very short time (say 2 weeks max). Work like refactoring, writing tests, improving build systems, etc would be just things you do every day and nobody would ever mention it. Of course, not everybody lives with an ideal situation, but I think most teams should strive to move their situation as close to this as they can (there are definite exceptions, though).
- std_throwaway 7y agoHere's what normal people see and hear when you enter the room: Nerd opens mouth: "Blablabla quibberish, you're all idiots, hear me express your problem way overcomplicated. Also, I need more budget and cannot promise to actually deliver a solution anytime soon because of more quibberish." If you want to be trusted start acting like normal people. Stop talking in technical quibberish. Stop behaving like a nerd. Shape up. Get some social interaction with normal people. Stop complaining all day long and start delivering what was asked on time and on budget.
- apatters 7y agoYes, business stakeholders often mistrust software and IT people, because relative to other industries we do a poor job of meeting our commitments. McKinsey did a study recently which found that large IT projects ran on average 45 percent over budget and delivered 56 percent less value than expected [1]. An executive is interacting with people from multiple industries or business functions and this is on the high side. It's much worse than a function like accounting, and it's also worse than many industries with bad reputations, like construction. So over time they are going to come to regard estimates from the tech guys with more suspicion than most. To the extent that other industries have this problem, yes you will find similar trust issues. If you are a PM/product manager then the best way to handle technical debt is to focus on the benefits that come from paying it down. You are not scheduling time for technical debt, you are scheduling time to improve reliability, security, scalability, etc. and from day one you are lobbying for more time to do every feature so that it's done right in the first place. Yes, sometimes one of these things has to go wrong before the business people start listening, but if you were calling it out from day one they will respect your input that much more. If you are a developer it's important to understand what work has business value and what doesn't, and make or inform decisions appropriately. Frankly there are developers out there whose attitude amounts to "Your job is to pay my salary, my job is to make whatever technical decisions I want without oversight, and if we don't deliver on time/on budget it's not my problem." Not all tech debt is worth paying down, the decision generally should be made based on the "interest rate" of the debt (how hard is this going to burn us in the long run?) and the term length of the "loan" (will this component be around for one year or ten?). [1] https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/delivering-large-scale-it-projects-on-time-on-budget-and-on-value https://www.mckinsey.com/business-functions/mckinsey-digital...
- watwut 7y ago> McKinsey did a study recently which found that large IT projects ran on average 45 percent over budget and delivered 56 percent less value than expected [1]. Is that fault of IT people or rather did business people over-promised? Because my repeated experience is that we give estimates, they are considered big and shorter estimates are sold. And it has nothing to do with IT people over-promising. I have yet to see a team where it is tech people pushing for shorter deadline and management being like "yeah, it is too risky, lets go with bigger estimate". There are situations of me making short estimate, but I also make estimates too big in other situations. However, somehow the estimate that makes its way toward contract is always short due to negotiating pressure. PM are all too often non-technical, it is quite unfair to blame "IT people" on mistakes of general management and project management. Project management blaming the these contracts being basically frauds on anyone else then themselves is literally them blaming scapegoats for their own faults.
- ernst_klim 7y ago> Do people in other professions have the same issues? Yes, any domain which doesn't produce a product, conceivable to a layman in one way or another. You need conceivable results to be understood by managers and other people. For example domains producing tangible results are doing much better. I used to be electronic engineer, and it was way simpler to explain laymen around you that you actually doing something. Blueprints, boards are conceivable more or less. Lines of code are not, especially hard to explain that less lines of code is better, since laymen don't see architecture, they see just text.
- stdbrouw 7y agoIt might have something to do with (1) how expensive it is to make software and (2) the fact that software engineers are known to sometimes be purists or aesthetes or susceptible to hype. A good comparison might be (building) architects: you know that if you let an architect run wild, there's a good chance you'll end up with something that is beautiful but wildly impractical and over budget. (And to a lesser extent, every expert wants to do a good job, not a good enough job.)
- EnderMB 7y agoI'm glad you've raised this, because it's something I've felt for a long time. A few friends of mine work in accredited professions, and it's not something they have to deal with all that much, most likely because they are all accredited. They are billed by the hour, and typically if someone isn't happy with the cost the response is "okay, we'll down tools". Inversely, companies that bid on contracts will bid with a fixed cost in mind, and when estimates are taken that go over the requirements things will be fudged to fit the requirement in for that price. This means one of three things: corners will be cut, developers have to work faster, or requirements are cut. In my experience, the last point is a rarity. I know that we software engineers like to think of ourselves as thought workers, and on the same level as the likes of lawyers and chartered accountants, but the way our industries operate are wildly different. I prefer to compare software engineering to a trade like carpentry, plumbing, etc. We share many similarities, such as easy barrier to entry and working on fixed-cost projects, but unlike trades, we're always willing to negotiate on our next pay day. You can say it's down to a difference in costs, but even on an industrial level where the person doing the sale isn't a trades-person, if a job requires £x and the client can only pay £y, then the requirements change or the job isn't taken on.
- repomies691 7y ago> The whole technical debt discussion often makes me wonder if there's a fundamental distrust (or even disrespect) of software engineers as domain experts. LOL, seriously? Guess you haven't worked that much with other practitioners, for example lawyers. There is some distrust always with business relationships, and I would say that it makes sense not to trust everyone 100% all the time. In the opposite, better not to trust everyone fully, even yourself.
- fabiocolasanti 7y agoThere is a widespread misunderstanding of what Technical Debt actually is and why it comes about. It's not about poor coding standards, it's a rational pursuit of commercial advantage through deferral of costs. I've written a few words about it a while ago here: https://datalanguage.com/features/reassessing-technical-debt https://datalanguage.com/features/reassessing-technical-debt