4 ms·

Being already known by my friends and family to be an "overthinker" and prone to "analysis paralysis", I sometimes wonder why I'm still a SWE. I've slowly start

Being already known by my friends and family to be an "overthinker" and prone to "analysis paralysis", I sometimes wonder why I'm still a SWE. I've slowly started to realize over time that my profession is filled with overthinking and over-engineering, and that our interview process can even select for it. IMO its hard to be the person that aces the technical interview gauntlet then walks out of the building and turns the analytical skills off. Many interview processes seem to favor how well a candidate can enumerate edge cases and problem spaces over effective risk assessment and cost management. They're both important to evaluate but often in practice the dumb solution is what my team ends up using because can be more maintainable, cheaper to build, easier to reason about, etc. Today my aim is to get my requirements, write as few lines of quality code in as short of a time as possible, test it, ship it and be done. Narrow focus and the ability to scope things down to what exactly what matters helps a lot. I defeat over-analysis by meditation, intentional dumbness/willful ignorance, and flow state.

amatecha 6y ago
As someone who's been programming for years, one of my biggest complaints in software is over-engineering. It's so universally prevalent, I literally expect to get downvoted for even making this comment. I strongly advocate for simplicity in software implementation, allowing for maintainability and flexibility... the cleanest solution that reaches the goal and doesn't lock us too far into a narrow approach that may be difficult to deviate from. Sometimes that's not realistic, but it's actually feasible a lot more than people seem to believe.
- qnt 6y ago
  Wholeheartedly agree. The best system is no system at all - problem is it doesn't really satisfy any requirements. Anything added beyond that is extra room where fragility creeps in & the tradeoff shouldn't ever be taken lightly.
- IndrekR 6y ago
  Reminds me this comment: https://news.ycombinator.com/item?id=19977678 https://news.ycombinator.com/item?id=19977678 There are other coping mechanisms you can use, although some of them are arguably worse than procrastination (eg overengineering).
- flippinburgers 6y ago
  I like to believe that experience will eventually drive people towards using scientific parsimony when creating solutions to engineering problems. I don't think over-engineering is the only problem: shiny-new. Shiny-new syndrome wherein the typically junior programmer cannot help but jump from one new thing to the next, advocating that some new system must use the new tech that they recently discovered. While the enthusiasm is ok, working with people like this can be a drag. Another problem: we are facebook too! In the sense that small companies heartily believe that they actually have problems at the scale of a company like facebook (or will ever get to that level). Again, hearing someone say, "We should do xyz because Netflix does it!" makes me want to give up on this profession.
  - xvilka 6y ago
    > Another problem: we are facebook too! In the sense that small companies heartily believe that they actually have problems at the scale of a company like facebook (or will ever get to that level). Precisely! It's tiresome how many people setup Kubernetes for a network of 2-6 computers.
    - deepsun 6y ago
      I was with you once, but last year's I got tired when people set up networks of 2-6 computers with daemons that run in a wildly different ways, and it's not clear how to fix their failures, where configs and metrics and logs are. I'd honestly prefer overengineered kubernetes setup nkw, only for some more standard way of doing things between projects.
      - majewsky 6y ago
        That's a false dichotomy though. Having a standard way of doing things between projects does not require running a complex distributed system with dozens of pieces all engaged in constant chatter. It's a testament to the failings of our profession that we apparently cannot agree on a sane baseline toolset less complicated than Kubernetes.
    - marcosdumay 6y ago
- feoren 6y ago
  It's a weird term, "over engineering". Think about over-engineering a bridge. What does it look like? Huge and bulky, way too expensive, able to hold much more weight than it should, lots of walkways and access points and design features that nobody really wants? But wait: "any fool can build a bridge that doesn't fall down. It takes an engineer to build a bridge that just barely doesn't fall down." The whole point of engineering is to figure out exactly what the bridge needs to do and make it do only that. So the "over-engineered" bridge is actually under-engineered -- they haven't thought enough about the actual problem. So too is most code that is called "over-engineered" -- it's far more common that people haven't though enough about the problem than that they've over-thought it.
  - amatecha 6y ago
    When I say "over-engineering" I mean stuff like this: https://github.com/MarkSFrancis/enterprise-fizz-buzz https://github.com/MarkSFrancis/enterprise-fizz-buzz Building a whole ton of stuff, just to do something fairly trivial.
    - JMTQp8lwXL 6y ago
      In this example, it's black and white obviously over engineered. The reality is, in an application of moderate complexity, whether or not it is indeed over-engineered is a far more subjective calculation. E.g., >90% of engineers would agree enterprise-fizz-buzz is over-engineered. But perhaps ~50% of engineers agree my SaaS app is over-engineered.
    - giantDinosaur 6y ago
      Just to be very clear, that application is intended as a joke. (just in case)
  - sh87 6y ago
    I like to call it "poor engineering" and there are many forms of poor engineering I've seen. Following the analogy... painting the bottom of the bridge in same color as the terrain. Then repainting because there were color differences. Spending time and money looking for lighter materials even when the bridge is designed to support a lot more weight than traditional time tested materials. Justification : "Maybe cars will get heavier, and bigger saving 3 pounds is a good pursuit"
- snarkypixel 6y ago
  "over-engineering" is a fuzzy term though. For someone, using a third-party service to handle users auth flow is a time saver, and building the whole flow themselves would be over-engineered. For another, using a third-party service for auth would be adding layers of complexity on top of a simple db request, and would thus be considered over-engineered.
- danybittel 6y ago
  "I would have written a shorter letter, but I did not have the time." In my experience, you often start with a somewhat over engineered solution. Then you take the (long) process of really understanding it and slowly distill it down. This is true for a new feature and also the whole codebase.
  - amatecha 6y ago
    Yeah, that's a great point. The final code I submit is often 2/3 the size of what I initially got running (perhaps less, even). I'm always happy to see a code review that is removing more code than it's adding, haha :) I like to use the phrase "measure twice, cut once" in regards to shipping code. Take the time to pare it down to the truly essential functionality, trimming away extraneous stuff that complicates things and may even be premature optimization. It's certainly a skill that takes time to improve at, haha
- rkagerer 6y ago
  Largely agree, but I've found distilling complex requirements down to a simple implementation takes effort, and sometimes longer than the more circuitous route. A coarse rule of thumb from a few decades of my own experience in software development is that version 1 will be quick, dirty and (in retrospect) naive. Version 2 will improve it with what you learned the first time around. And version 3 tends to be the clean, solid, polished winner. I've learned to expect and encourage refactoring to arrive at this point.
- kylen 6y ago
  In my opinion over-engineering is a symptom of not enough thinking. Complex, over engineered solutions don't happen because a problem is too well understood, they happen because a problem is not understood well enough. This is based on the assumption that more thinking == better understanding.
incog_nit0 6y ago
Recently I've been interviewing and assessing technical tests regularly. We have this one technical coding challenge. Most of the submissions are 1000+ LOC and quite heavily engineered. One submission though was 300 LOC, runs 3x as quick as everyone elses and is the only one to get 100% in our acceptance tests. The author was very self-deprecating about it - describing it as a quickly cobbled together submission. I'm nearly 40 and find I over analyze the design of everything. Which is great when I'm architecting a high level software feature, but when I get to coding I'm almost at analysis paralysis over every, damn detail. I miss that sense of flow.
- bobthepanda 6y ago
  Generally speaking, the less complicated something is, the less weird combinations of states one can find themselves in. Now you probably shouldn‘t code golf either, but KISS works as an operating principle for a reason.
fnord77 6y ago
"if it's dumb and it works, it's not dumb"
- thaumasiotes 6y ago
  "You can't argue with success."
magnusmundus 6y ago
> the ability to scope things down to what exactly what matters In the last few years, I started shifting my focus from what variety of cases to handle with an elaborate solution, to what possible cases I might be blocking with a simple solution. Gratuitous YAGNI was an important intermediate step in that process. It might not be very FAANG-friendly (or it might not scale to four-digits-engineers regardless of brand), but both in terms of real life performance and peer feedback, it's yielded very good results. While the first is prone to becoming a bottomless pit of SWE-self-pleasuring; the latter is actually very useful in coming up with something that's cheap, easy and maintainable, but still flexible enough -- which doesn't necessarily mean extensible! Ease-of-refactoring is another flavor of flexibility, and that's where I found this kind of thinking beating YAGNI alone. One actual drawback I see is that it makes DRY difficult to achieve, but the more people I work with, the more I'm becoming disillusioned with that anyway. It's such an overly adaptable dogma to generate work that is often pointless, and becomes harmful so easily. I think we found so much comfort in "don't type the same thing twice" (though I still try to keep it to 3) that we don't stop to think about "don't maintain the same information twice" anymore. But this is 3AM rambling, so I'll stop here.
BurningFrog 6y ago
> I sometimes wonder why I'm still a SWE. I've slowly started to realize over time that my profession is filled with overthinking and over-engineering You've already seen how to be a better SWE than most. Now you just have to execute.

amatecha 6y ago

As someone who's been programming for years, one of my biggest complaints in software is over-engineering. It's so universally prevalent, I literally expect to get downvoted for even making this comment. I strongly advocate for simplicity in software implementation, allowing for maintainability and flexibility... the cleanest solution that reaches the goal and doesn't lock us too far into a narrow approach that may be difficult to deviate from. Sometimes that's not realistic, but it's actually feasible a lot more than people seem to believe.

qnt 6y ago
Wholeheartedly agree. The best system is no system at all - problem is it doesn't really satisfy any requirements. Anything added beyond that is extra room where fragility creeps in & the tradeoff shouldn't ever be taken lightly.
IndrekR 6y ago
Reminds me this comment: https://news.ycombinator.com/item?id=19977678 https://news.ycombinator.com/item?id=19977678 There are other coping mechanisms you can use, although some of them are arguably worse than procrastination (eg overengineering).
flippinburgers 6y ago
I like to believe that experience will eventually drive people towards using scientific parsimony when creating solutions to engineering problems. I don't think over-engineering is the only problem: shiny-new. Shiny-new syndrome wherein the typically junior programmer cannot help but jump from one new thing to the next, advocating that some new system must use the new tech that they recently discovered. While the enthusiasm is ok, working with people like this can be a drag. Another problem: we are facebook too! In the sense that small companies heartily believe that they actually have problems at the scale of a company like facebook (or will ever get to that level). Again, hearing someone say, "We should do xyz because Netflix does it!" makes me want to give up on this profession.
- xvilka 6y ago
  > Another problem: we are facebook too! In the sense that small companies heartily believe that they actually have problems at the scale of a company like facebook (or will ever get to that level). Precisely! It's tiresome how many people setup Kubernetes for a network of 2-6 computers.
  - deepsun 6y ago
    I was with you once, but last year's I got tired when people set up networks of 2-6 computers with daemons that run in a wildly different ways, and it's not clear how to fix their failures, where configs and metrics and logs are. I'd honestly prefer overengineered kubernetes setup nkw, only for some more standard way of doing things between projects.
    - majewsky 6y ago
      That's a false dichotomy though. Having a standard way of doing things between projects does not require running a complex distributed system with dozens of pieces all engaged in constant chatter. It's a testament to the failings of our profession that we apparently cannot agree on a sane baseline toolset less complicated than Kubernetes.
  - marcosdumay 6y ago
feoren 6y ago
It's a weird term, "over engineering". Think about over-engineering a bridge. What does it look like? Huge and bulky, way too expensive, able to hold much more weight than it should, lots of walkways and access points and design features that nobody really wants? But wait: "any fool can build a bridge that doesn't fall down. It takes an engineer to build a bridge that just barely doesn't fall down." The whole point of engineering is to figure out exactly what the bridge needs to do and make it do only that. So the "over-engineered" bridge is actually under-engineered -- they haven't thought enough about the actual problem. So too is most code that is called "over-engineered" -- it's far more common that people haven't though enough about the problem than that they've over-thought it.
- amatecha 6y ago
  When I say "over-engineering" I mean stuff like this: https://github.com/MarkSFrancis/enterprise-fizz-buzz https://github.com/MarkSFrancis/enterprise-fizz-buzz Building a whole ton of stuff, just to do something fairly trivial.
  - JMTQp8lwXL 6y ago
    In this example, it's black and white obviously over engineered. The reality is, in an application of moderate complexity, whether or not it is indeed over-engineered is a far more subjective calculation. E.g., >90% of engineers would agree enterprise-fizz-buzz is over-engineered. But perhaps ~50% of engineers agree my SaaS app is over-engineered.
  - giantDinosaur 6y ago
    Just to be very clear, that application is intended as a joke. (just in case)
- sh87 6y ago
  I like to call it "poor engineering" and there are many forms of poor engineering I've seen. Following the analogy... painting the bottom of the bridge in same color as the terrain. Then repainting because there were color differences. Spending time and money looking for lighter materials even when the bridge is designed to support a lot more weight than traditional time tested materials. Justification : "Maybe cars will get heavier, and bigger saving 3 pounds is a good pursuit"
snarkypixel 6y ago
"over-engineering" is a fuzzy term though. For someone, using a third-party service to handle users auth flow is a time saver, and building the whole flow themselves would be over-engineered. For another, using a third-party service for auth would be adding layers of complexity on top of a simple db request, and would thus be considered over-engineered.
danybittel 6y ago
"I would have written a shorter letter, but I did not have the time." In my experience, you often start with a somewhat over engineered solution. Then you take the (long) process of really understanding it and slowly distill it down. This is true for a new feature and also the whole codebase.
- amatecha 6y ago
  Yeah, that's a great point. The final code I submit is often 2/3 the size of what I initially got running (perhaps less, even). I'm always happy to see a code review that is removing more code than it's adding, haha :) I like to use the phrase "measure twice, cut once" in regards to shipping code. Take the time to pare it down to the truly essential functionality, trimming away extraneous stuff that complicates things and may even be premature optimization. It's certainly a skill that takes time to improve at, haha
rkagerer 6y ago
Largely agree, but I've found distilling complex requirements down to a simple implementation takes effort, and sometimes longer than the more circuitous route. A coarse rule of thumb from a few decades of my own experience in software development is that version 1 will be quick, dirty and (in retrospect) naive. Version 2 will improve it with what you learned the first time around. And version 3 tends to be the clean, solid, polished winner. I've learned to expect and encourage refactoring to arrive at this point.
kylen 6y ago
In my opinion over-engineering is a symptom of not enough thinking. Complex, over engineered solutions don't happen because a problem is too well understood, they happen because a problem is not understood well enough. This is based on the assumption that more thinking == better understanding.

incog_nit0 6y ago

Recently I've been interviewing and assessing technical tests regularly. We have this one technical coding challenge. Most of the submissions are 1000+ LOC and quite heavily engineered. One submission though was 300 LOC, runs 3x as quick as everyone elses and is the only one to get 100% in our acceptance tests. The author was very self-deprecating about it - describing it as a quickly cobbled together submission. I'm nearly 40 and find I over analyze the design of everything. Which is great when I'm architecting a high level software feature, but when I get to coding I'm almost at analysis paralysis over every, damn detail. I miss that sense of flow.

bobthepanda 6y ago
Generally speaking, the less complicated something is, the less weird combinations of states one can find themselves in. Now you probably shouldn‘t code golf either, but KISS works as an operating principle for a reason.

fnord77 6y ago

"if it's dumb and it works, it's not dumb"

thaumasiotes 6y ago
"You can't argue with success."

magnusmundus 6y ago

> the ability to scope things down to what exactly what matters In the last few years, I started shifting my focus from what variety of cases to handle with an elaborate solution, to what possible cases I might be blocking with a simple solution. Gratuitous YAGNI was an important intermediate step in that process. It might not be very FAANG-friendly (or it might not scale to four-digits-engineers regardless of brand), but both in terms of real life performance and peer feedback, it's yielded very good results. While the first is prone to becoming a bottomless pit of SWE-self-pleasuring; the latter is actually very useful in coming up with something that's cheap, easy and maintainable, but still flexible enough -- which doesn't necessarily mean extensible! Ease-of-refactoring is another flavor of flexibility, and that's where I found this kind of thinking beating YAGNI alone. One actual drawback I see is that it makes DRY difficult to achieve, but the more people I work with, the more I'm becoming disillusioned with that anyway. It's such an overly adaptable dogma to generate work that is often pointless, and becomes harmful so easily. I think we found so much comfort in "don't type the same thing twice" (though I still try to keep it to 3) that we don't stop to think about "don't maintain the same information twice" anymore. But this is 3AM rambling, so I'll stop here.

BurningFrog 6y ago

> I sometimes wonder why I'm still a SWE. I've slowly started to realize over time that my profession is filled with overthinking and over-engineering You've already seen how to be a better SWE than most. Now you just have to execute.