4 ms·
Technical debt is an invisible $1.52T problem
- bookofjoe 3y agohttps://archive.ph/5taMT https://archive.ph/5taMT
- 2devnull 3y ago“The only lasting fix for technical debt is for management to empower the IT side of an organization to start saying no to all the requests for shiny new things, and focus on making what an organization already has more robust, more secure and futureproof.” One could argue that technical debt is really just regular debt. If can see that, then you also will see that federal fiscal and monetary policy are really the long term fix, not IT departments saying “No” which of course they already do.
- thfuran 3y agoThe whole problem is that it isn't. If it were regular debt, it would be visible on balance sheets instead of just ruining everything behind the scenes.
- amelius 3y agoPerhaps we just need better bookkeeping ;)
- zer00eyz 3y agoValue accounting? Like cost per user for infra? operating cost per hour, by hour? like that... You need an accountant embedded in your department to just watch and track spend and map it to something. Preferably they are in tech and told to hold product teams feet to the fire.
- kevindamm 3y agoWell, businesses in the US are now required to account for R&E (including improvements to existing software) as capital investment.. Where should I put the line items for technical debt, and how should I quantify it?
- infinghxsg 3y ago[dead]
- ldjkfkdsjnv 3y agoBusiness people continue to collect large paychecks while technical debt ruins their companies ability to deliver innovative product. There are no repercussions, and no mechanism to correct this. There's also no easy visibility into the fact that this is occurring, aside from the mental strain on junior/mid level engineers. Even at the principle engineer level, the incentive is to continue collecting paychecks and not rock the boat. There are many exceptions among technically sophisticated companies, but your average old school company is really a vessel for corporate careerists to build personal wealth. But all of the above assume that technical debt should be fixed, alot of the time its a huge waste of resources to fix with minimal business value. Most money flowing into a company comes from a few systems, everything else is not worth it.
- elmerfud 3y agoWhat's wrong with me viewing the business as a way for me to personally build wealth? I want to do a good job and I want to do the best of my abilities but I cannot care more than the management infrastructure does. The entire purpose of the corporation is to build wealth for their shareholders and the board so what incentive do I have that they do not have? How can I be more invested in the company than the owners and leadership of that company? Maybe it wasn't your intention but it seems like your post is blaming the engineers for the problem rather than the people who allocate the engineers time. The engineering staff is just operating according to the demonstrated core company principles. Of which those poor principles are usually greed.
- ldjkfkdsjnv 3y agoWhen high level management optimizes for personal wealth we end up with companies like Boeing.
- cebert 3y agoI’ve been in the software industry for over 16 years now. From my experiences, it seems like it’s always more difficult to get investment in addressing technical debt. First, many developers, me included, aren’t always the best at communicating the value investing in scoped improvements projects brings to the business. I’ve improved at communicating the value upwards over time. Executives don’t see software the same way engineers do and you need to learn to speak their language. Second, investing in a new feature or project can easily become a more desirable investment because it’s new, exciting, and potentially a differentiator. The best way to get investment to address tech debt seems to have a crisis unless organizations have a deliberate, systematic process the helps avoid accumulating too much debt.
- nextworddev 3y agoUnpopular opinion but tech management likes having tech debt, as it persists and justifies their org structure by keeping the problems around.
- avg_dev 3y agoi'd lean more to: most management does not understand how building software works
- nextworddev 3y agothey know enough to know that solving tech debt doesn’t help their cause
- sshine 3y ago> many developers, me included, aren’t always the best at communicating the value investing in scoped improvements projects brings to the business While that is true, I don’t think it could be a communication-from-developers problem. Tech debt impairs development speed and causes small irritations in the every-day use of the product. In my experience, what works is extreme ownership. Someone with agency has to care.
- agentultra 3y ago
- oneplane 3y agoIs it really invisible, or is it just not shown? I've seen plenty of environments where the technical debt stares you in the face wherever you look, but no reports make it to decision makers (and the ones to make decisions are generally not the ones in the position to see all that debt). If you're in an org pretending everything is fine, that would probably apply to many more problem topics and not just technical debt.
- cebert 3y agoDoes that boil down to organization culture and whether lower-level employees feel comfortable raising issues to leadership? I believe Boeing would be in a much better situation if it encouraged raising concerns instead of suppressing them.
- oneplane 3y agoI suppose it is often a combination of company culture, local culture and KPIs and then a bunch of smaller things. I'm not up to speed on the latest and greatest research, but last time I went into that rabbit hole it boiled down to hiding problems or moving them around so the KPIs look good (be it KPIs in the company, or in the case of publicly traded companies, the 'virtual' KPIs of news and press releases), and the degree to which hierarchy forces everything down. That directionality gets exponentially worse when you have a local culture of strong hierarchy, and a company culture of making leadership look good (or honour culture?). I've only seen that personally from a slight distance where it really looks like a kind of dictatorship where management comes up with an idea, and the subordinates have to make due with it, and only positive results may be shared or they get fired. The ideas are usually bad or really hard to execute so you see people use all available resources to present 'success' with not much left for the actual thing that needed doing.
- moomoo11 3y agoJOB SECURITY SECURED Initiative: CREATE MORE TECH DEBT RESOLVING THE TECH DEBT
- foota 3y agoI wonder if it would ever make sense for technical debt to be recognized via accounting.
- d0gsg0w00f 3y agoI believe it is in a way. Everywhere I've ever worked in tech leadership has a dataset of applications marked as decom, maintain, or invest. They never plan on doing anything with "maintain" until a replacement comes along. There are hundreds of thousands of saints toiling daily on maintain and decom apps. We need all types to make the world work. There are trucking companies driving old trucks that break down all the time that are total accounting write-offs. They have competitors with shiny new leased fleets. Both situations are known to accounting and reported on. It's a trade off of what leadership does with that info. Is it better to have no expenses and still bring in occasional revenue or is it better to have reliable revenue with high expenses and low margin? Not always an easy decision but definitely not one that resides in accounting.
- gamepsys 3y ago> Years ago, companies owned their own servers, and the penalties for inefficient systems weren’t that great—just buy some more racks of computers and throw those at the problem. Now that everything has shifted, or is shifting, to the cloud, and companies are paying for computing a la carte, those inefficiencies mean bills that are, on average, double what they could be if old systems were optimized, says Ben DeBow, chief executive of Fortified, a consulting firm that helps companies manage and update their database systems. I've found good success in the past with associating my optimizations directly into the cloud bill. This looks like "Improved efficiency of {MyService} by optimizing {cache hit rate}, reducing the number of pods required of the service by 25%, saving {$300k/year} in AWS costs." I look for these types of wins every time I am on a new team.
- singingfish 3y agoThe other week I saved my company a grand a year by reusing an existing service for some contractors rather than some new cloud service. Then promptly expensed a big chunk of that for repairs on my office chair :)
- dboreham 3y agoRepairs?
- zer00eyz 3y agoIf you walk into a job and cut their cost by 25 percent are you a hero or a snitch. Candidly everyone in your leadership between you and the CFO/CEO is going to think you just made them look bad. And you did. It is easier to keep a bad decision hidden than it is to admit you fucked up, or worse that you burned a pile of money.
- datascienced 3y agoThat is if they let you cut that cost. They may only let a top 5 firm consultant do that after paying their firm a few million.
- eastbound 3y ago- Developers leave after 3 years, - Developers are vastly overrated. They keep developing new frameworks when they’d better spend their time solving business problems. Given their cost, no-one dares telling them anything. - Can’t motivate them to do tech debt. - They always suggest to refactor everything. It’s basically impossible to give business sense to a developer. Otherwise they wouldn’t be developers! But they want to be free. It’s not a “My boss told me not to do tech debt.” It’s “Every time I’m given free time to tackle tech debt, I open Draw.io to create charts of what I suggest as a new infrastructure with microkubes.”
- seattle_spring 3y agoHmm that's not how I would describe engineers I work with at all. What sort of companies are you seeing the majority of devs act like this?
- ungreased0675 3y agoI suspect it was an exaggeration to make a point. As an engineer manager, I do agree with the sentiment though. Keeping engineers focused is tough. Allowing them time to burn down tech debt might mean they go off and work on interesting projects instead of something of business value. I’ve also been burned by the “let’s start over and do it right” refactor that resulted in no customer value. Making software is full of tradeoffs.
- oneplane 3y agoThis looks like a description of a team or department that has no product owners and no requirements elicitation process. Quality aspects can be measured, and resulting requirements will mitigate technical debt, unless you put that analysis so far down the backlog that it falls off. But all of that is management or 'team overhead' work.
- dboreham 3y agoMicrokubes.
- rvnx 3y agoTechnical debt is like financial debt: it's not a problem if it is kept under control, and it may even be a very smart leverage; it has a limited liability in many ways. Also innovation and debt are often intermingled. SpaceX cutting corners, it could be seen as debt by slow-paced companies, but at the end of the day, the result is here.
- hayst4ck 3y agoThe only fix is quality forward thinking details oriented leadership. Given that corporate America has decided that all employees are meant to be replaceable and therefore transient, it is not rational for any individual to orient themselves to the long term or make near term concessions. Likewise if employee compensation is a function of hours worked more than it is a function of the company's success, it is irrational for an employee to make decisions with respect to the long term, especially if they will be rewarded for speed over complexity tradeoffs and they can leave the company with a great resume when the debt becomes overwhelming. Admiral Rickover's speech Doing a Job speaks quite a bit to technical debt and the forces that promote it: https://govleaders.org/rickover.htm https://govleaders.org/rickover.htm
- echelon 3y agoAnother good fix is legacy companies getting replaced by newer, more nimble companies. It's good for the economy for new startups to tackle old problems in new and novel ways, throwing out decades of legacy cruft and ossification. It also puts more comp into the hands of the founders and frontier employees that pave the way rather than funneling it to institutional shareholders of an old business - pension funds, etc.
- datascienced 3y agoStartups are engines of new tech debt.
- nuc1e0n 3y agoSure, but for companies that fail their code ceases to be run by them. If their code is used by other groups then they will have to deal with maintenance themselves, if that's even possible. This risk is somewhat mitigated by demanding only open source software be used. At least then if a software's supplier goes bankrupt then updated versions can still be produced. That's not the case with closed source software.
- 3y ago
- siliconc0w 3y agoThe problem is that all business incentives are based on short-term outcomes. Sometimes there is a culture of being deliberate about tech-debt but that culture eventually gets eroded by the MBA/bean counters, poor hiring, or scaling too fast. The end result is you end up pouring ever increasing amounts into engineering for smaller and smaller deltas of change. Eventually to the point it can become 'terminal' where you literally cannot change the system in one place without breaking it in another. At that point you just sell the company if you can to some PE firm that will load it up with debt, loot it, and dump the corpse to decompose in the nearest body of water.
- nuc1e0n 3y agoTech debt isn't always a bad thing if it is managed properly. Developing for the short term can help a company to succeed in the market until a better solution can be produced, where more careful work may have prevented the survival of the company. The problem comes when technical debt is never paid off and continually accumulates, even when it's not necessary for it to.
- Supermancho 3y agoTech debt that fails fast or allows the company runway to fund a better design down the road, is a market advantage. Short term thinking is more effective, which is why it's so common. Skip some considerations to figure out the specific target audience or functionality sooner. I don't think the name fits at all. It's tech centric, which is self serving of the tech community, by the tech community.
- NomDePlum 3y agoThe biggest problem with the term "technical debt" is that it's thrown about by too many people for too many reasons. Mostly well meaning reasons, but often highly personalized and with literally no intrinsic value or thought attached. More often than not it's used to express a personal preference, rather than a guiding principle. I've seldom seen it used positively, to a point I don't actually believe it has any benefit as a term and given that I don't think it actually exists. It's not an actionable term. Talk about and make cases for system performance, operability, functionality, security, cost, support knowledge and similar and you'll have much easier value based conversations that have a chance of having continued buy in long term from your organisation.
- deleted 3y ago[deleted]
- georgelyon 3y agoOne thing I’ve noticed in my career is that it is often quite difficult to distinguish “good” technical infrastructure improvements (meaning ones that will actually help a company achieve its goals) from “bad” ones (stuff that is unnecessary, risky or simply not worth the investment). I've seen these decisions made more using the political capital of high level engineers than by anything data-driven. That’s not to say this isn’t a problem, just maybe more of an art than a science.
- jvans 3y agoA lot of things I've seen engineers do in the name of cleaning up technical debt actively make things worse Left to their own devices a lot of engineers will go to enormous lengths for either marginal improvements or changes that actively hurt. I can understand why leadership is skeptical of these investments because even for someone extremely technical, if you're not in the weeds everyday of that particular problem, it's very hard to know what's worth it
- nickjj 3y agoI really wish businesses thought more about the butterfly effect. Putting small tickets that help reduce technical debt, increase developer happiness or help make progress towards a longer term technical goal at the bottom of the backlog because something larger and product focused always gets higher priority is such a great way to ensure you end up with lots of technical debt and unhappy teams of developers. It is demoralizing to continue to identify problems and outline solutions but they never see the light of day. Also when you encounter the problem again and again and again it's reliving this negative memory of "well this is never going to change". Eventually this happens long enough where it becomes a culture at the company and most folks stop investing their energy into identifying them and things get worse as time goes as you cycle through new developers because people quit and new hires come in. The company loses so much with the loss of domain knowledge and hiring that the product greatly suffers in the end. Never under estimate how much of a difference a motivated and happy team can make.
- bastard_op 3y ago"Please enable JS and disable any ad blocker" No.
- a13o 3y agoThe term 'technical debt' is conversationally bankrupt. Every possible programming anxiety is technical debt. Don't understand the system, the historical constraints, or the business objectives when it was written? Slap a 'technical debt' sticker on it and now you're the smartest person in the room again. If you want to be an effective roadmap participant, learn to phrase technical problems in terms of immediate benefit to customers. It can then at least be prioritized against the reams of other immediate customer benefit ideas in the backlog. If you want to marshal an impressive development cadence, focus far upstream of 'technical debt'. Whatever that term means to you, it's a lagging indicator. The solution is procedural, not technical.
- yarekt 3y agoThis, 1000%. I’ve never had any luck with figuring out a way to get management on board with tech debt, and this is a nimble startup with smart people… solution is elsewhere, not “write more quality software” near the symptoms.
- nextworddev 3y agoYour comment is refreshingly honest and cutting to the gist of things. Wish I had more colleagues like you.
- 000ooo000 3y agoThis dismissal of 'tech debt' as a phrase/concept based on weak strawman examples is just a contrarian rant followed by LinkedIn-esque advice that seems ignorant of the fact that hyper focus on 'immediate benefit' is almost always the cause of.. the technical problems that should be phrased in terms of their immediate benefit, or something.
- uuddlrlrbaba 3y agoHi, it looks like you're writing a reply. Try using sentences -Clippy
- etagwerker 3y agoI'm glad this topic if finally getting the attention it needs. I really like Google's paper on technical debt: https://ieeexplore.ieee.org/document/10109339 https://ieeexplore.ieee.org/document/10109339 I wrote an article with a summary of the key insights in that paper: https://www.ombulabs.com/blog/tech-debt-maturity-model.html https://www.ombulabs.com/blog/tech-debt-maturity-model.html Hopefully, the more articles like this one, the more non-technical managers will finally give a shit about technical debt?
- williamcotton 3y agoSigh… if you want to communicate the issue with business minded individuals in a position of leadership it helps to use the correct accounting terminology. “Technical debt” is a misnomer. Software is an asset. If it is made quickly and shoddily it amortizes/depreciates at a faster rate than otherwise. Weighing maintenance costs against recapitalization is the conversation to have. There is clearly a way to make measurements that can help answer the question of what direction to take.
- 1vuio0pswjnm7 3y agoWorks where archive.ph is blocked: https://cc.bingj.com/cache.aspx?d=4904741369235685&w=4jnumCxrX1C2aze2mjy6VKjsl1THnjQW https://cc.bingj.com/cache.aspx?d=4904741369235685&w=4jnumCx...
- smackeyacky 3y agoIt’s a bullshit description that is thrown around too much. Working code is good code and if it ain’t broken don’t fuck with it.
- deleted 3y ago[deleted]