6 ms·

The reason I bring this up is I am surprised that architectural choice is overwhelmingly the top technical debt concern. That doesn't seem right. I would expect

The reason I bring this up is I am surprised that architectural choice is overwhelmingly the top technical debt concern. That doesn't seem right. I would expect it to be poor code due to time constraints. 1. The system grew alongside the devs' understanding of the problem space. There was never time to change decisions that were later found to be not the best (and the older such a decision is, the harder it is to fix). 2. The problem space changes over time. Either because the outside world changes, or because of scope or scale increases. This means that even good architectural decisions can become bad over time. Isolated pieces of bad code can be fixes when/if they need to be touched for some other reason. Fixing currently-inappropriate architecture takes a bit more effort.

shoo 11y ago
This is very wise. You try to make the best architectural decision that you can at the time, with the knowledge and resources you have available. Time passes and you learn new things -- maybe the problem changes, or your understanding of it improves, or your understanding of alternative implementation strategies improves. For whatever reason, you can now imagine a new architecture that would be superior, if only it were implemented to replace the old architecture. Now you have technical debt. This doesn't necessarily mean that the best decision now is to pay off the debt by reworking the architecture -- that depends upon a cost/benefit/opportunity cost analysis. edit: Tangentially, there's a pretty interesting presentation by Kevlin Henney titled "The Architecture of Uncertainty" [1]. My poor summary: When designing the initial architecture of a system, Kevlin suggests that the team brainstorm to identify which parts of the system have a lot of uncertainty. Each region of uncertainty then becomes a subsystem. Put interfaces between the subsystems that need to be connected. Hopefully you now have an architecture with stable interfaces, even if individual subsystems need to be completely rewritten during the course of the project. [1] http://www.infoq.com/presentations/Architecture-Uncertainty http://www.infoq.com/presentations/Architecture-Uncertainty
- kctess5 11y ago
  Disclaimer I haven't watched the presentation, though I do have first hand experience refactoring large systems that grew slowly over time. Based off of your edit's description, this technique feels like a patch that inevitably fall apart. It relies on the assumption that your team can correctly identify the centers of uncertainty, and that that uncertainty model will continue to apply. Thinking about such things is an excellent idea, but it is not sufficient - uncertainty is, after all, uncertain. I think in many cases the more important thing is to create an abstraction that allows you to perfectly represent your existing business logic in the most concise way possible. This should let you cut down on the number of edge cases outside the model, and generally simplify the system. Simplicity in specification is important, because it will allows newcomers to quickly understand the inner workings of your code, quickly correlating business logic with real code - if they can understand it, and can work within it, then they will not be tempted to hack around it (which is the root of code deterioration). I strongly believe that human friendliness and understandability should be key design goals in ANY new system, not an after thought. So long as no one breaks the abstraction, the 99%, day to day changes should be easy. When you finally do hit a case that requires a significant abstraction change, then your concise code will make it obvious that it's outside of your abstraction model, and can evaluate options at that point.
mikekchar 11y ago
The thing that is very frustrating in the "There was never time to change decisions..." bit is that there often is time. If you are curious about what I mean, I invite you to try a little experiment. For a few weeks (2-4 should be sufficient) tell people, "If you see something wrong with the design, refactor it as soon as you see it. If you need some extra time on your story to compensate for that work, just bring it up at standup and we will modify the sprint commitment". If your experience is like mine, you will find that the vast majority of people will not refactor the design at all. About 10% of the people will try to refactor something, and will end up trying to rewrite the entire app. They will do more damage than good and will probably give up half way through. If you are lucky, maybe 5% will actually refactor something and be successful. Because I have tried this many times, I've interviewed people and asked them why they do what they do. For the people who don't refactor, the reason they usually give is: "There is no time to refactor". Which is really odd because they have explicitly been given time. What I have come to realize, though, is that people do not want time; they want absolution of responsibility. If you ask them to make the judgement call, they do the math in their head (unconsciously) and determine that they will be rewarded more and criticised less if they do feature work without refactoring. This forces the decision up to the PM/PGM/BM/BA/Whatever, for whom refactoring has no direct benefit. The result is that refactoring is rarely done, and if it is done it is the result of a large political process. For the people who try to rewrite everything, training seems to be the overwhelming issue. They pull on a thread and the whole sweater comes apart. For some people in this category, though, giving them carte blanche to decide what to do means that they feel they can finally "do it right". Doing it right in this context means that they can replace all the code that they didn't personally write and therefore don't like. Since everyone on the team only writes a small portion of the code, it means that rewriting everybody else's portion is basically rewriting the app. Again, training seems to help because even if a person's goal is to replace everything, if they learn how to do it piece by piece they can be successful. Also if they do that, they will be required to have many coversations and may eventually learn how to work with others. Finally, you may get one or two people who naturally know how to refactor well. It is useful to find out who these people are and to encourage them. Unfortunately, this often enrages the "my way or the highway" people. The people who are good at refactoring, if encouraged, will naturally dominate the design of the application. Often these people are suppressed by political means because they are so effective at driving the design. In order to enable these people you will need to make some tough decisions on the business end of things.
makeitsuckless 11y ago
I have found the #1 source of architecture/reality mismatches over time being that "business" has deliberately kept developers only partially informed on a "need to know" basis. To be clear, there was nothing malicious about it, it's just that many stakeholders only give their short term needs as input, not the long term strategy that's discussed behind closed doors in the board room. The big problem here is that non-engineers don't get that some strategies don't simply add to the problem space, but fundamentally change it. "Why the fuck didn't you tell us sooner?" is one of the most common phrases in software development.
- davidgerard 11y ago
  Quite a lot of the Scrum process is the valiant attempt to extract coherent requirements from an end user. You should be so lucky as for them to have a coherent plan at all, let alone a secret one. Insofar as they do have one, in my experience there's always a few bits with magical flying unicorn ponies as a requirement they'll get to in time.

shoo 11y ago

This is very wise. You try to make the best architectural decision that you can at the time, with the knowledge and resources you have available. Time passes and you learn new things -- maybe the problem changes, or your understanding of it improves, or your understanding of alternative implementation strategies improves. For whatever reason, you can now imagine a new architecture that would be superior, if only it were implemented to replace the old architecture. Now you have technical debt. This doesn't necessarily mean that the best decision now is to pay off the debt by reworking the architecture -- that depends upon a cost/benefit/opportunity cost analysis. edit: Tangentially, there's a pretty interesting presentation by Kevlin Henney titled "The Architecture of Uncertainty" [1]. My poor summary: When designing the initial architecture of a system, Kevlin suggests that the team brainstorm to identify which parts of the system have a lot of uncertainty. Each region of uncertainty then becomes a subsystem. Put interfaces between the subsystems that need to be connected. Hopefully you now have an architecture with stable interfaces, even if individual subsystems need to be completely rewritten during the course of the project. [1] http://www.infoq.com/presentations/Architecture-Uncertainty http://www.infoq.com/presentations/Architecture-Uncertainty

kctess5 11y ago
Disclaimer I haven't watched the presentation, though I do have first hand experience refactoring large systems that grew slowly over time. Based off of your edit's description, this technique feels like a patch that inevitably fall apart. It relies on the assumption that your team can correctly identify the centers of uncertainty, and that that uncertainty model will continue to apply. Thinking about such things is an excellent idea, but it is not sufficient - uncertainty is, after all, uncertain. I think in many cases the more important thing is to create an abstraction that allows you to perfectly represent your existing business logic in the most concise way possible. This should let you cut down on the number of edge cases outside the model, and generally simplify the system. Simplicity in specification is important, because it will allows newcomers to quickly understand the inner workings of your code, quickly correlating business logic with real code - if they can understand it, and can work within it, then they will not be tempted to hack around it (which is the root of code deterioration). I strongly believe that human friendliness and understandability should be key design goals in ANY new system, not an after thought. So long as no one breaks the abstraction, the 99%, day to day changes should be easy. When you finally do hit a case that requires a significant abstraction change, then your concise code will make it obvious that it's outside of your abstraction model, and can evaluate options at that point.

mikekchar 11y ago

The thing that is very frustrating in the "There was never time to change decisions..." bit is that there often is time. If you are curious about what I mean, I invite you to try a little experiment. For a few weeks (2-4 should be sufficient) tell people, "If you see something wrong with the design, refactor it as soon as you see it. If you need some extra time on your story to compensate for that work, just bring it up at standup and we will modify the sprint commitment". If your experience is like mine, you will find that the vast majority of people will not refactor the design at all. About 10% of the people will try to refactor something, and will end up trying to rewrite the entire app. They will do more damage than good and will probably give up half way through. If you are lucky, maybe 5% will actually refactor something and be successful. Because I have tried this many times, I've interviewed people and asked them why they do what they do. For the people who don't refactor, the reason they usually give is: "There is no time to refactor". Which is really odd because they have explicitly been given time. What I have come to realize, though, is that people do not want time; they want absolution of responsibility. If you ask them to make the judgement call, they do the math in their head (unconsciously) and determine that they will be rewarded more and criticised less if they do feature work without refactoring. This forces the decision up to the PM/PGM/BM/BA/Whatever, for whom refactoring has no direct benefit. The result is that refactoring is rarely done, and if it is done it is the result of a large political process. For the people who try to rewrite everything, training seems to be the overwhelming issue. They pull on a thread and the whole sweater comes apart. For some people in this category, though, giving them carte blanche to decide what to do means that they feel they can finally "do it right". Doing it right in this context means that they can replace all the code that they didn't personally write and therefore don't like. Since everyone on the team only writes a small portion of the code, it means that rewriting everybody else's portion is basically rewriting the app. Again, training seems to help because even if a person's goal is to replace everything, if they learn how to do it piece by piece they can be successful. Also if they do that, they will be required to have many coversations and may eventually learn how to work with others. Finally, you may get one or two people who naturally know how to refactor well. It is useful to find out who these people are and to encourage them. Unfortunately, this often enrages the "my way or the highway" people. The people who are good at refactoring, if encouraged, will naturally dominate the design of the application. Often these people are suppressed by political means because they are so effective at driving the design. In order to enable these people you will need to make some tough decisions on the business end of things.

makeitsuckless 11y ago

I have found the #1 source of architecture/reality mismatches over time being that "business" has deliberately kept developers only partially informed on a "need to know" basis. To be clear, there was nothing malicious about it, it's just that many stakeholders only give their short term needs as input, not the long term strategy that's discussed behind closed doors in the board room. The big problem here is that non-engineers don't get that some strategies don't simply add to the problem space, but fundamentally change it. "Why the fuck didn't you tell us sooner?" is one of the most common phrases in software development.

davidgerard 11y ago
Quite a lot of the Scrum process is the valiant attempt to extract coherent requirements from an end user. You should be so lucky as for them to have a coherent plan at all, let alone a secret one. Insofar as they do have one, in my experience there's always a few bits with magical flying unicorn ponies as a requirement they'll get to in time.