4 ms·

The problem with over-engineered projects... our team inherited a project that was designed by some architects in an ivory tower and implemented by some other e

The problem with over-engineered projects... our team inherited a project that was designed by some architects in an ivory tower and implemented by some other engineers. It was quite complex (message queues, multi-threading, async etc.) so that it would be 'scalable' yet crushed under the slightest load, customers were experiencing delays during peak hours. We removed about a quarter of the code (some intermittent queues & components), nothing changed in terms of functionality. Now we're removing more code and switching over to less complex data structures to fix the delays. Maybe a complete rewrite would have been better after all, who knows. I'm not a very experienced developer. I like to ship stuff and get projects done in reasonable time and a "YAGNI" attitude. Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. On the other hand, I see a tendency that if you give projects (it could be a greenfield project or some maintenance) to a team of, say, 6 developers, then they will just 'create work' for themselves, filling up the todo list with items which may lead to an over-engineered project that could have been easily be done by 3 people within the same timeframe. If you have some good reads/stories on this topic, patterns to watch for, I'd appreciate if you shared them.

majewsky 6y ago
> Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. This is a very good attitude, but I would like to tack on a second part to it: Instead of designing your code to be infinitely extensible in all directions, make it easily replaceable instead. It's impossible to anticipate every future requirement, but certain choices will make it easier to deal with a new requirement as it pops up. One of these things is unit tests. If you have some gnarly business logic with tons of edge cases, you really really want a good test suite because it enables you to add new edge cases with confidence.
- ljm 6y ago
  I'm a strong proponent of this attitude. The most maintainable code is that you can easily delete. It does require a lot more thought and discipline, because it's not just a case of constructing an epic hierarchy of abstractions, or stubbing/mocking the hell out of your code in the tests to swap things around.
- suzzer99 6y ago
  Couldn't agree more. A little redundancy early on in a product's lifecycle, when requirements are still in their infancy, can be a very valuable thing. It's much easier to add a layer of abstraction later as requirements became clear, vs. remove an unneeded layer of abstraction (which is often impossible). I've been on too many projects which tried to build the perfect abstraction right out of the box - as if the problem domain fully known.
thechao 6y ago
I was where you’re at 15 years ago; my language of choice was C++/Boost. I was advised to try C, not because it is a good language, but because it’s so hard to use, it helps you focus on the exact problem you need to solve. To this day I always “do one in C; do one in X”. I think Python (substitute any ‘batteries included’ language) can be good for this, too, as there’s no need to Yak shave with them.
tomtomtom1 6y ago
I genuinely hate the software architect title. It's like when we were in school and the teacher told us to write an outline before writing. No problem ! the majority wrote the text first and then the outline. That's because people in general write to think.* The level of deep thought achieved through writing is harder to achieve beforehand. In the same sense, the level of deep thought achieved through writing code, is hard to achieve beforehand. It's actually worse because your engineers, dangled in this web of classes, won't be able to think about the big picture for themselves. An Architect thinks in terms of what sounds good, what is beautiful in OOP-land not in terms of what is easy to write, and what's performant to implement. One poignant example is the interviewer who wants you to implement chess pieces as classes. which sounds good in the architects' world but is blatantly insane if you think about the actual code and the actual challenges you're facing. * writing to think and writing to be read are conflicting goals that why editors exist.

majewsky 6y ago

> Let's just rewrite stuff and introduce more abstractions when it actually hurts and a refactor is in scope. This is a very good attitude, but I would like to tack on a second part to it: Instead of designing your code to be infinitely extensible in all directions, make it easily replaceable instead. It's impossible to anticipate every future requirement, but certain choices will make it easier to deal with a new requirement as it pops up. One of these things is unit tests. If you have some gnarly business logic with tons of edge cases, you really really want a good test suite because it enables you to add new edge cases with confidence.

ljm 6y ago
I'm a strong proponent of this attitude. The most maintainable code is that you can easily delete. It does require a lot more thought and discipline, because it's not just a case of constructing an epic hierarchy of abstractions, or stubbing/mocking the hell out of your code in the tests to swap things around.
suzzer99 6y ago
Couldn't agree more. A little redundancy early on in a product's lifecycle, when requirements are still in their infancy, can be a very valuable thing. It's much easier to add a layer of abstraction later as requirements became clear, vs. remove an unneeded layer of abstraction (which is often impossible). I've been on too many projects which tried to build the perfect abstraction right out of the box - as if the problem domain fully known.

thechao 6y ago

I was where you’re at 15 years ago; my language of choice was C++/Boost. I was advised to try C, not because it is a good language, but because it’s so hard to use, it helps you focus on the exact problem you need to solve. To this day I always “do one in C; do one in X”. I think Python (substitute any ‘batteries included’ language) can be good for this, too, as there’s no need to Yak shave with them.

tomtomtom1 6y ago

I genuinely hate the software architect title. It's like when we were in school and the teacher told us to write an outline before writing. No problem ! the majority wrote the text first and then the outline. That's because people in general write to think.* The level of deep thought achieved through writing is harder to achieve beforehand. In the same sense, the level of deep thought achieved through writing code, is hard to achieve beforehand. It's actually worse because your engineers, dangled in this web of classes, won't be able to think about the big picture for themselves. An Architect thinks in terms of what sounds good, what is beautiful in OOP-land not in terms of what is easy to write, and what's performant to implement. One poignant example is the interviewer who wants you to implement chess pieces as classes. which sounds good in the architects' world but is blatantly insane if you think about the actual code and the actual challenges you're facing. * writing to think and writing to be read are conflicting goals that why editors exist.