4 ms·

I think you're on the right track when you're saying that this isn't a technical problem. That's the most important thing to understand about the development of

I think you're on the right track when you're saying that this isn't a technical problem. That's the most important thing to understand about the development of overwrought, top-heavy system architectures. Trying to fix this with more code is a fool's errand. As soon as you get something reasonable and good, someone will come along to distort and abuse it to serve their non-technical interests. IMO a few of the main reasons we see this plague of listless over-engineering are: a) many people don't actually know what they're doing, and they're trying to hide behind ever-shifting technical targets to obscure this fact. Sometimes this is pretty blatant, but this happens in a more subtle form with surprising frequency. Many people maintain only a very superficial interest and understanding in what's happening under the covers, so they hear promises of magic hand-waving and instantly jump to them. and b) people are forced by the principal to coop themselves up in an office for 8 hours per day, and they need to invent things to occupy their time. You could say that this is the agent not respecting the principal's investment, but it really goes both ways. Our employment model does not allow for "done", because as soon as this happens, the contributor is declared redundant and the revenue stream disappears. It's fun to brainstorm models that may not have that issue. A relatively flat/reasonable technical apparatus is needed to reason coherently about a system at all. A lot of these microservice shops are complete nightmares, and they use "microservices" as cover for "nothing is maintainable, just delete it and start over when we need to make an important change". People primarily want to use the latest stacks and make complex system architectures to gratify their vanity, in various ways: to keep their resume relevant, to be treated like a guru, and to keep up with the joneses. These are problems that stretch way outside of tech and affect everything humans do. Have other technical fields found ways of handling this sanely? What is stopping civil engineers from changing to a different concrete formulation every two weeks? Is that there is an objective physical component that can be tested, whereas software lives primarily as a manifestation of a mental problem-solving structure? Those are the answers that our field needs to look for IMO.

majormajor 9y ago
> and b) people are forced by the principal to coop themselves up in an office for 8 hours per day, and they need to invent things to occupy their time. You could say that this is the agent not respecting the principal's investment, but it really goes both ways. Our employment model does not allow for "done", because as soon as this happens, the contributor is declared redundant and the revenue stream disappears. It's fun to brainstorm models that may not have that issue. I think you might be overstating this part. I haven't been in many positions where if I said "hey, I finished that thing you asked me to do, and it took five fewer people and six fewer months than we thought" the response would be "ok, looks like we can fire you all now" instead of "great! get started on the next from the list of 100 things we think might be useful for the business!" + some good results at performance review time. My usual frustration is the opposite: I can't knock out my tasks as quickly as I could because I'm waiting on other teams to figure out how to add an unexpected new feature to their overengineered buzzwordy codebase. Maybe this is a high-growth vs low-growth environment type of thing, but even in more stable businesses, seems like the maintenance backlog never really empties. Just gotta make sure the bosses understand that you wouldn't fire the workers every time the shop temporarily clears - you're still going to need them for the next jobs that come in.
- cookiecaper 9y ago
  I think that a lot of the motive in this direction is unconscious. It's less "Oh boy, they'll realize we aren't doing anything if we don't say we need to rewrite something", and more, "This will be an interesting way to spend my time and keep myself busy for a while." This is the same impulse, just vocalized differently. If the incentive structure was changed, devs may rather think "I need to make this as tight, small, and maintainable as possible, so that I can spend the rest of my time on the beach." We see this a lot more on side projects specifically because the dev knows and respects that the available resources are constrained, as the parent poster mentions. In any case, people usually won't allow it to reach the inflection point where they're visibly non-productive and the boss is cognizant of the option to "clear the shop". There are always tweaks, defects and flaws that can be made up and justified. The question is whether that work is really contributing useful value. Once a project reaches a certain level of maturity, this becomes increasingly dubious.
- Chiba-City 9y ago
  That's your team's problem. Optimizing single developers for "getting their part over" is not contributing. Software delivery is not like taking tests in school. It's team delivery over multiple release. The only wins are group delivery.
closeparen 9y ago
>nothing is maintainable, just delete it and start over when we need to make an important change Good! That's why we have microservices, classes, or interchangeable parts in any engineering discipline: rather than laboriously extricating a deeply integrated, tightly coupled component (or worse, give up and throw away the whole system), we get to rip out one specific component, throw it away, and replace it with something that conforms to the same interface while keeping the rest. If you can routinely delete and replace specific services, your microservices architecture is working beautifully. Just as, when you can routinely overhaul specific classes, your OO design is working well. In the physical world, maintenance and repair hardly ever consist of coaxing the same matter into working state. It's extracting old parts, throwing them away, and putting in new ones.
- cookiecaper 9y ago
  > In the physical world, maintenance and repair hardly ever consist of coaxing the same matter into working state. It's extracting old parts, throwing them away, and putting in new ones. The difference is what you consider "the same matter". Since software exists in the realm of the mind, you have to come to an explicit consensus; there is no natural or physical barrier. A mechanic will allow dirty oil to drain, but they won't rip out your oil reservoir and redo all your hoses every time you want to get your oil replaced. That'd be very expensive and wasteful. It is not good to use parts that are so cheap that they tear whenever someone pokes or nudges them. That'd make it so no one could move around without getting lots of oil all over the place. Basic maintenance should be possible. In almost all cases, you should not export every function as its own independent microservice. You should not have to track a request across 60 internal services when you're trying to debug. Yet, this is what we've gotten from naive microservice implementers all too often. In a microservice architecture, many of the tools that would otherwise validate and verify a successful refactoring are unavailable, because dependent code is exported and accessed over HTTP instead of language-level constructs that the compiler or other static analysis tools could check. In any architecture, it is rare for a programmer to be able to gut a dependency without causing collateral damage somewhere down the chain. Somehow, this policy of "throw out everything related to SERVICE_X and start over whenever we need to make change" doesn't seem very practical...
- 9y ago

majormajor 9y ago

> and b) people are forced by the principal to coop themselves up in an office for 8 hours per day, and they need to invent things to occupy their time. You could say that this is the agent not respecting the principal's investment, but it really goes both ways. Our employment model does not allow for "done", because as soon as this happens, the contributor is declared redundant and the revenue stream disappears. It's fun to brainstorm models that may not have that issue. I think you might be overstating this part. I haven't been in many positions where if I said "hey, I finished that thing you asked me to do, and it took five fewer people and six fewer months than we thought" the response would be "ok, looks like we can fire you all now" instead of "great! get started on the next from the list of 100 things we think might be useful for the business!" + some good results at performance review time. My usual frustration is the opposite: I can't knock out my tasks as quickly as I could because I'm waiting on other teams to figure out how to add an unexpected new feature to their overengineered buzzwordy codebase. Maybe this is a high-growth vs low-growth environment type of thing, but even in more stable businesses, seems like the maintenance backlog never really empties. Just gotta make sure the bosses understand that you wouldn't fire the workers every time the shop temporarily clears - you're still going to need them for the next jobs that come in.

cookiecaper 9y ago
I think that a lot of the motive in this direction is unconscious. It's less "Oh boy, they'll realize we aren't doing anything if we don't say we need to rewrite something", and more, "This will be an interesting way to spend my time and keep myself busy for a while." This is the same impulse, just vocalized differently. If the incentive structure was changed, devs may rather think "I need to make this as tight, small, and maintainable as possible, so that I can spend the rest of my time on the beach." We see this a lot more on side projects specifically because the dev knows and respects that the available resources are constrained, as the parent poster mentions. In any case, people usually won't allow it to reach the inflection point where they're visibly non-productive and the boss is cognizant of the option to "clear the shop". There are always tweaks, defects and flaws that can be made up and justified. The question is whether that work is really contributing useful value. Once a project reaches a certain level of maturity, this becomes increasingly dubious.
Chiba-City 9y ago
That's your team's problem. Optimizing single developers for "getting their part over" is not contributing. Software delivery is not like taking tests in school. It's team delivery over multiple release. The only wins are group delivery.

closeparen 9y ago

>nothing is maintainable, just delete it and start over when we need to make an important change Good! That's why we have microservices, classes, or interchangeable parts in any engineering discipline: rather than laboriously extricating a deeply integrated, tightly coupled component (or worse, give up and throw away the whole system), we get to rip out one specific component, throw it away, and replace it with something that conforms to the same interface while keeping the rest. If you can routinely delete and replace specific services, your microservices architecture is working beautifully. Just as, when you can routinely overhaul specific classes, your OO design is working well. In the physical world, maintenance and repair hardly ever consist of coaxing the same matter into working state. It's extracting old parts, throwing them away, and putting in new ones.

cookiecaper 9y ago
> In the physical world, maintenance and repair hardly ever consist of coaxing the same matter into working state. It's extracting old parts, throwing them away, and putting in new ones. The difference is what you consider "the same matter". Since software exists in the realm of the mind, you have to come to an explicit consensus; there is no natural or physical barrier. A mechanic will allow dirty oil to drain, but they won't rip out your oil reservoir and redo all your hoses every time you want to get your oil replaced. That'd be very expensive and wasteful. It is not good to use parts that are so cheap that they tear whenever someone pokes or nudges them. That'd make it so no one could move around without getting lots of oil all over the place. Basic maintenance should be possible. In almost all cases, you should not export every function as its own independent microservice. You should not have to track a request across 60 internal services when you're trying to debug. Yet, this is what we've gotten from naive microservice implementers all too often. In a microservice architecture, many of the tools that would otherwise validate and verify a successful refactoring are unavailable, because dependent code is exported and accessed over HTTP instead of language-level constructs that the compiler or other static analysis tools could check. In any architecture, it is rare for a programmer to be able to gut a dependency without causing collateral damage somewhere down the chain. Somehow, this policy of "throw out everything related to SERVICE_X and start over whenever we need to make change" doesn't seem very practical...
9y ago