4 ms·

My comment in that discussion from ~15 years ago was based on experiences with a project that I "inherited" - 30,000 Java classes and interfaces, 20+ layers of

My comment in that discussion from ~15 years ago was based on experiences with a project that I "inherited" - 30,000 Java classes and interfaces, 20+ layers of abstraction, team of 30+ people working for a long time. And most importantly, it didn't actually work! I actually "finished" it to the point of a working system for the customer with real customers in about 5 weeks by ignoring most of what they had done - required a handful of classes and JSPs. Probably distinctly under-architected but it worked and was built on for a number of years after.

gombosg 7y ago
The problem with over-engineered projects... our team inherited a project that was designed by some architects in an ivory tower and implemented by some other engineers. It was quite complex (message queues, multi-threading, async etc.) so that it would be 'scalable' yet crushed under the slightest load, customers were experiencing delays during peak hours. We removed about a quarter of the code (some intermittent queues & components), nothing changed in terms of functionality. Now we're removing more code and switching over to less complex data structures to fix the delays. Maybe a complete rewrite would have been better after all, who knows. I'm not a very experienced developer. I like to ship stuff and get projects done in reasonable time and a "YAGNI" attitude. Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. On the other hand, I see a tendency that if you give projects (it could be a greenfield project or some maintenance) to a team of, say, 6 developers, then they will just 'create work' for themselves, filling up the todo list with items which may lead to an over-engineered project that could have been easily be done by 3 people within the same timeframe. If you have some good reads/stories on this topic, patterns to watch for, I'd appreciate if you shared them.
- majewsky 7y ago
  > Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. This is a very good attitude, but I would like to tack on a second part to it: Instead of designing your code to be infinitely extensible in all directions, make it easily replaceable instead. It's impossible to anticipate every future requirement, but certain choices will make it easier to deal with a new requirement as it pops up. One of these things is unit tests. If you have some gnarly business logic with tons of edge cases, you really really want a good test suite because it enables you to add new edge cases with confidence.
  - ljm 7y ago
    I'm a strong proponent of this attitude. The most maintainable code is that you can easily delete. It does require a lot more thought and discipline, because it's not just a case of constructing an epic hierarchy of abstractions, or stubbing/mocking the hell out of your code in the tests to swap things around.
  - suzzer99 7y ago
    Couldn't agree more. A little redundancy early on in a product's lifecycle, when requirements are still in their infancy, can be a very valuable thing. It's much easier to add a layer of abstraction later as requirements became clear, vs. remove an unneeded layer of abstraction (which is often impossible). I've been on too many projects which tried to build the perfect abstraction right out of the box - as if the problem domain fully known.
- thechao 7y ago
  I was where you’re at 15 years ago; my language of choice was C++/Boost. I was advised to try C, not because it is a good language, but because it’s so hard to use, it helps you focus on the exact problem you need to solve. To this day I always “do one in C; do one in X”. I think Python (substitute any ‘batteries included’ language) can be good for this, too, as there’s no need to Yak shave with them.
- tomtomtom1 7y ago
  I genuinely hate the software architect title. It's like when we were in school and the teacher told us to write an outline before writing. No problem ! the majority wrote the text first and then the outline. That's because people in general write to think.* The level of deep thought achieved through writing is harder to achieve beforehand. In the same sense, the level of deep thought achieved through writing code, is hard to achieve beforehand. It's actually worse because your engineers, dangled in this web of classes, won't be able to think about the big picture for themselves. An Architect thinks in terms of what sounds good, what is beautiful in OOP-land not in terms of what is easy to write, and what's performant to implement. One poignant example is the interviewer who wants you to implement chess pieces as classes. which sounds good in the architects' world but is blatantly insane if you think about the actual code and the actual challenges you're facing. * writing to think and writing to be read are conflicting goals that why editors exist.
bane 7y ago
Similar experience here. About 6 months ago I inherited a team of 30 that had spent 4 years building some monstrous microservice "thing" that didn't work and never seemed to build reliably and barely worked even if it could be deployed. Lots and lots of late nights trying to "deploy to testing" and wasted weekends restarting things. I stopped the work, broke the team up (and let a few of them go), re-assigned the goals of the project to 3 people who, working half-time, from scratch, and in Python, rebuilt the entire thing in 3 months, delivered it and it worked (and continues to work). "Builds" are now an scp to a VM, and a restart of a flask application. My wife recalls a similar experience where a complex application she worked on (and ran reliably on very little infrastructure and with an entire team of 4), and serviced thousands of simultaneous users, was replaced (when a new CTO came in) by a giant, consultant driven Java project that required approximately 6x the infrastructure, 3x the staff to keep it going, and at launch could only serve 9 to 12 users at once. It was a debacle. But sunk cost fallacy forced her company to throw millions of dollars of consultants at it until it achieved some minimal level of barely available service. She and her boss left around that time. She heard that a couple years later the CTO was let go, the consultants fired and it was all replaced by a team of 5 rewriting it all again from scratch (in Java) but like you also without all the architecture barf dropped in from orbit, and immediately went back to servicing the thousands of users again. I'm convinced that this is the fault of some kind of consultant industry that seems to infest enterprise software circles that is really only good at inventing ways of designing systems that require more of their services, but never seem to actually deliver working systems.
- nurettin 7y ago
  As someone who fell into a similar situation just last year, I can say that it helps a lot when you have previous work that had been done before you were involved. Perhaps the existing project doesn't show you how things should have been done, but it certainly shows you some of the challenges that were faced and what not to do about them so you don't fall into the same traps as they did. So it's not always "those architecture astronaut guys used the company's resources to study their webscale fantasies and I came to save the day", but rather, "they did as well as they could using the choices and resources they stuck with and I happened upon the project with 20/20 hindsight".

gombosg 7y ago

The problem with over-engineered projects... our team inherited a project that was designed by some architects in an ivory tower and implemented by some other engineers. It was quite complex (message queues, multi-threading, async etc.) so that it would be 'scalable' yet crushed under the slightest load, customers were experiencing delays during peak hours. We removed about a quarter of the code (some intermittent queues & components), nothing changed in terms of functionality. Now we're removing more code and switching over to less complex data structures to fix the delays. Maybe a complete rewrite would have been better after all, who knows. I'm not a very experienced developer. I like to ship stuff and get projects done in reasonable time and a "YAGNI" attitude. Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. On the other hand, I see a tendency that if you give projects (it could be a greenfield project or some maintenance) to a team of, say, 6 developers, then they will just 'create work' for themselves, filling up the todo list with items which may lead to an over-engineered project that could have been easily be done by 3 people within the same timeframe. If you have some good reads/stories on this topic, patterns to watch for, I'd appreciate if you shared them.

majewsky 7y ago
> Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. This is a very good attitude, but I would like to tack on a second part to it: Instead of designing your code to be infinitely extensible in all directions, make it easily replaceable instead. It's impossible to anticipate every future requirement, but certain choices will make it easier to deal with a new requirement as it pops up. One of these things is unit tests. If you have some gnarly business logic with tons of edge cases, you really really want a good test suite because it enables you to add new edge cases with confidence.
- ljm 7y ago
  I'm a strong proponent of this attitude. The most maintainable code is that you can easily delete. It does require a lot more thought and discipline, because it's not just a case of constructing an epic hierarchy of abstractions, or stubbing/mocking the hell out of your code in the tests to swap things around.
- suzzer99 7y ago
  Couldn't agree more. A little redundancy early on in a product's lifecycle, when requirements are still in their infancy, can be a very valuable thing. It's much easier to add a layer of abstraction later as requirements became clear, vs. remove an unneeded layer of abstraction (which is often impossible). I've been on too many projects which tried to build the perfect abstraction right out of the box - as if the problem domain fully known.
thechao 7y ago
I was where you’re at 15 years ago; my language of choice was C++/Boost. I was advised to try C, not because it is a good language, but because it’s so hard to use, it helps you focus on the exact problem you need to solve. To this day I always “do one in C; do one in X”. I think Python (substitute any ‘batteries included’ language) can be good for this, too, as there’s no need to Yak shave with them.
tomtomtom1 7y ago
I genuinely hate the software architect title. It's like when we were in school and the teacher told us to write an outline before writing. No problem ! the majority wrote the text first and then the outline. That's because people in general write to think.* The level of deep thought achieved through writing is harder to achieve beforehand. In the same sense, the level of deep thought achieved through writing code, is hard to achieve beforehand. It's actually worse because your engineers, dangled in this web of classes, won't be able to think about the big picture for themselves. An Architect thinks in terms of what sounds good, what is beautiful in OOP-land not in terms of what is easy to write, and what's performant to implement. One poignant example is the interviewer who wants you to implement chess pieces as classes. which sounds good in the architects' world but is blatantly insane if you think about the actual code and the actual challenges you're facing. * writing to think and writing to be read are conflicting goals that why editors exist.

bane 7y ago

Similar experience here. About 6 months ago I inherited a team of 30 that had spent 4 years building some monstrous microservice "thing" that didn't work and never seemed to build reliably and barely worked even if it could be deployed. Lots and lots of late nights trying to "deploy to testing" and wasted weekends restarting things. I stopped the work, broke the team up (and let a few of them go), re-assigned the goals of the project to 3 people who, working half-time, from scratch, and in Python, rebuilt the entire thing in 3 months, delivered it and it worked (and continues to work). "Builds" are now an scp to a VM, and a restart of a flask application. My wife recalls a similar experience where a complex application she worked on (and ran reliably on very little infrastructure and with an entire team of 4), and serviced thousands of simultaneous users, was replaced (when a new CTO came in) by a giant, consultant driven Java project that required approximately 6x the infrastructure, 3x the staff to keep it going, and at launch could only serve 9 to 12 users at once. It was a debacle. But sunk cost fallacy forced her company to throw millions of dollars of consultants at it until it achieved some minimal level of barely available service. She and her boss left around that time. She heard that a couple years later the CTO was let go, the consultants fired and it was all replaced by a team of 5 rewriting it all again from scratch (in Java) but like you also without all the architecture barf dropped in from orbit, and immediately went back to servicing the thousands of users again. I'm convinced that this is the fault of some kind of consultant industry that seems to infest enterprise software circles that is really only good at inventing ways of designing systems that require more of their services, but never seem to actually deliver working systems.

nurettin 7y ago
As someone who fell into a similar situation just last year, I can say that it helps a lot when you have previous work that had been done before you were involved. Perhaps the existing project doesn't show you how things should have been done, but it certainly shows you some of the challenges that were faced and what not to do about them so you don't fall into the same traps as they did. So it's not always "those architecture astronaut guys used the company's resources to study their webscale fantasies and I came to save the day", but rather, "they did as well as they could using the choices and resources they stuck with and I happened upon the project with 20/20 hindsight".