4 ms·

i worked at a company a while back that decided it was time to refactor due to application complexity, which was driving developer productivity down significant

i worked at a company a while back that decided it was time to refactor due to application complexity, which was driving developer productivity down significantly a team of elite coders and a senior architect was put in charge of the project, and they decided to implement IoC & OSGi throughout the code base and split the repos up into cleaner modules a year and a half later, they were done. glowing company wide emails were sent, it was a new era things got significantly worse: the new system was very difficult to understand, testing cross-module issues was impossible to coordinate, OSGi added almost no value at a heavy cost across the code base, etc. developer productivity, already low, fell even further the senior architect left shortly after the project was completed careful with that refactor, eugene

ptr 5y ago
Would be interesting to hear more about this — why did they think the new way would improve the situation? Why was the new solution harder to work with? Why was it hard to understand?
- tartoran 5y ago
  Wrong refactoring? Added new unnecessary complexity? I’ve seen refactoring gone wrong, not specifically OSGi, as I have no idea what it really is.
gonzo41 5y ago
Companies exploit their workers, and sometimes workers exploit their companies by selling them a dream and leveling up to a new gig. I would be wary of hiring someone who came off a project like that who bailed from a position just as things were delivered.
- dntrkv 5y ago
  Of course that could be the case, but in my experience , the inverse is way more common. A team is tasked with rearchitecting some part of a system, and by the time they’re done, everyone is burnt-out and one-after-another the engineers leave the company. The goal was never to “level up”, it’s just that these large deliverables take a lot out of you.
- ma2rten 5y ago
  It actually seems like a good sign to me that they completed their current project before jumping ship.
- chinchilla2020 5y ago
  Leaving during the middle of the project would be worse. This is an attrition issue. The company lost an employee, fair game.
- tartoran 5y ago
  I’ve seen some of that as well. After a very large refactoring is done it is initially claimed a success by the lead but to only slowly dawn on the rest of the team that the codebase is not better, just more complicated without any significant gains. Mess is still there, it’s just a unfamiliar one. Questions to the lead of the refactoring project cannot be answered as they are long gone some other place.
almostdeadguy 5y ago
I'll argue that things like this have nothing to do w/ the merits of refactoring in general and are usually due to poor project management. A small team orchestrating a large scale refactor in isolation for over a year seems appealing to project managers because it allows that team to focus without interrupting the pace of feature development, but it makes it exceedingly likely that the output of that effort will fail for these reasons: 1. Evolving requirements. A refactor is a design that is focused on solving challenges _at that time_. Things happen over a year. A product grows, its supported feature set requires rethinking the nature of the domain, the pressures of scale force new considerations, teams get more rigorous about the kind of automated testing they practice, etc. A team that doesn't have a feedback loop into is likely to build something that may have worked for the system of a year and half ago, but no longer makes sense. 2. Lack of buy-in and knowledge sharing. In isolation, most software engineers tend to solve their own problems. Senior engineers often make assumptions about the knowledge of the rest of the team and about the coherency of their abstractions when they don't have to teach it to anyone or battle test it in any meaningful way. This is why collaborative processes like code review and pairing are often a huge win, but sharing these efforts needs to go farther and a team should be pushed to try incremental adoption and feedback until there's more confidence about a refactor. Large scale refactors should be chunked and interspersed w/ a workload that attempts to validate the success of the design (i.e. working alongside teams doing feature development to make use of the refactored design). And perhaps most importantly: the team working on this needs to have clarity on what it means to be successful. How do they know that their proposed design is meeting the needs of the team at large? Infrastructure, dev ex, or any team that is orchestrating something like this needs to think about their task like a product team, and "what does success look like" needs to be a question from the onset.
- recursivedoubts 5y ago
  problem w/ that is that "is refactoring a good idea" becomes unfalsifiable: you can always blame something else this is sort of like w/ the agile folks: when an agile implementation fails, "oh, you didn't do agile right" obviously, better code is always desirable, but the temptation to rewrite is very strong in developers (I feel it all the time) and experience suggests it often leads first to a worse situation, and then a worser one
hedora 5y ago
I've noticed organizations with lots of tech debt promote exactly the people that created the problem in the first place. Giving that group a big budget and autonomy to fix the issue is like paying tobacco company to cure your lung cancer.
- dilyevsky 5y ago
  That’s right if someone shipped a buggy unreadable codebase the first time what makes anyone think they are actually capable of doing the opposite second time around?
- Jtsummers 5y ago
  The people who create a lot of tech debt (and other problems) are often highly visible and appear to Get Things Done. In fact, they may actually get things done, but in a poor fashion (in this context). Repenning & Sterman, Nobody Ever Gets Credit for Fixing Problems that Never Happened [PDF] https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.pdf https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.... (pretty sure this is the one I want) Being proactive and doing the right thing (maybe, potential higher costs earlier than they can be supported for instance) is a low-visibility activity. Even if it's your job to do the cleanup work, the one who wrote it first is the one most likely to get the credit no matter the quality. Because they got something out there for customers, not you.
  - ZephyrBlu 5y ago
    A funny comic describing this exact issue: https://twitter.com/shreyas/status/1391594520207233024 https://twitter.com/shreyas/status/1391594520207233024.
umvi 5y ago
The rewrite should be done by the people that will be maintaining it, not "elite coders" that aren't going to touch it ever again afterwards
- hyperman1 5y ago
  Almost every split between maintainers and creators will kill your productivity stone dead a week after delivery. Not only have you guaranteed total loss of knowledge, someone who had never maintained anything has no chance of writing code that survives contact with production. You'll get all sorts of shiny frameworks and nice architectural dreams, but no logging, no monitoring, no validation of the actual user requirements, no nothing. You spend the next 5 years rewriting from scratch if you're lucky.

ptr 5y ago

Would be interesting to hear more about this — why did they think the new way would improve the situation? Why was the new solution harder to work with? Why was it hard to understand?

tartoran 5y ago
Wrong refactoring? Added new unnecessary complexity? I’ve seen refactoring gone wrong, not specifically OSGi, as I have no idea what it really is.

gonzo41 5y ago

Companies exploit their workers, and sometimes workers exploit their companies by selling them a dream and leveling up to a new gig. I would be wary of hiring someone who came off a project like that who bailed from a position just as things were delivered.

dntrkv 5y ago
Of course that could be the case, but in my experience , the inverse is way more common. A team is tasked with rearchitecting some part of a system, and by the time they’re done, everyone is burnt-out and one-after-another the engineers leave the company. The goal was never to “level up”, it’s just that these large deliverables take a lot out of you.
ma2rten 5y ago
It actually seems like a good sign to me that they completed their current project before jumping ship.
chinchilla2020 5y ago
Leaving during the middle of the project would be worse. This is an attrition issue. The company lost an employee, fair game.
tartoran 5y ago
I’ve seen some of that as well. After a very large refactoring is done it is initially claimed a success by the lead but to only slowly dawn on the rest of the team that the codebase is not better, just more complicated without any significant gains. Mess is still there, it’s just a unfamiliar one. Questions to the lead of the refactoring project cannot be answered as they are long gone some other place.

almostdeadguy 5y ago

I'll argue that things like this have nothing to do w/ the merits of refactoring in general and are usually due to poor project management. A small team orchestrating a large scale refactor in isolation for over a year seems appealing to project managers because it allows that team to focus without interrupting the pace of feature development, but it makes it exceedingly likely that the output of that effort will fail for these reasons: 1. Evolving requirements. A refactor is a design that is focused on solving challenges _at that time_. Things happen over a year. A product grows, its supported feature set requires rethinking the nature of the domain, the pressures of scale force new considerations, teams get more rigorous about the kind of automated testing they practice, etc. A team that doesn't have a feedback loop into is likely to build something that may have worked for the system of a year and half ago, but no longer makes sense. 2. Lack of buy-in and knowledge sharing. In isolation, most software engineers tend to solve their own problems. Senior engineers often make assumptions about the knowledge of the rest of the team and about the coherency of their abstractions when they don't have to teach it to anyone or battle test it in any meaningful way. This is why collaborative processes like code review and pairing are often a huge win, but sharing these efforts needs to go farther and a team should be pushed to try incremental adoption and feedback until there's more confidence about a refactor. Large scale refactors should be chunked and interspersed w/ a workload that attempts to validate the success of the design (i.e. working alongside teams doing feature development to make use of the refactored design). And perhaps most importantly: the team working on this needs to have clarity on what it means to be successful. How do they know that their proposed design is meeting the needs of the team at large? Infrastructure, dev ex, or any team that is orchestrating something like this needs to think about their task like a product team, and "what does success look like" needs to be a question from the onset.

recursivedoubts 5y ago
problem w/ that is that "is refactoring a good idea" becomes unfalsifiable: you can always blame something else this is sort of like w/ the agile folks: when an agile implementation fails, "oh, you didn't do agile right" obviously, better code is always desirable, but the temptation to rewrite is very strong in developers (I feel it all the time) and experience suggests it often leads first to a worse situation, and then a worser one

hedora 5y ago

I've noticed organizations with lots of tech debt promote exactly the people that created the problem in the first place. Giving that group a big budget and autonomy to fix the issue is like paying tobacco company to cure your lung cancer.

dilyevsky 5y ago
That’s right if someone shipped a buggy unreadable codebase the first time what makes anyone think they are actually capable of doing the opposite second time around?
Jtsummers 5y ago
The people who create a lot of tech debt (and other problems) are often highly visible and appear to Get Things Done. In fact, they may actually get things done, but in a poor fashion (in this context). Repenning & Sterman, Nobody Ever Gets Credit for Fixing Problems that Never Happened [PDF] https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.pdf https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.... (pretty sure this is the one I want) Being proactive and doing the right thing (maybe, potential higher costs earlier than they can be supported for instance) is a low-visibility activity. Even if it's your job to do the cleanup work, the one who wrote it first is the one most likely to get the credit no matter the quality. Because they got something out there for customers, not you.
- ZephyrBlu 5y ago
  A funny comic describing this exact issue: https://twitter.com/shreyas/status/1391594520207233024 https://twitter.com/shreyas/status/1391594520207233024.

umvi 5y ago

The rewrite should be done by the people that will be maintaining it, not "elite coders" that aren't going to touch it ever again afterwards

hyperman1 5y ago
Almost every split between maintainers and creators will kill your productivity stone dead a week after delivery. Not only have you guaranteed total loss of knowledge, someone who had never maintained anything has no chance of writing code that survives contact with production. You'll get all sorts of shiny frameworks and nice architectural dreams, but no logging, no monitoring, no validation of the actual user requirements, no nothing. You spend the next 5 years rewriting from scratch if you're lucky.