7 ms·

Over my career, I've worked with engineers that like to over-engineer and under-engineer. The over-engineered code looked like russian dolls: had many layers t

Over my career, I've worked with engineers that like to over-engineer and under-engineer. The over-engineered code looked like russian dolls: had many layers to it, and some of the abstractions offered no value. That can make onboarding to such code unnecessarily complex. On the other hand, under-engineered code made very little of use of even simple data structures or algorithms. I like to call it "chicken scratch" code. I find it tends to be brittle, and it fails the longevity test: you end up frequently having to touch everything, and be aware of the entire state of the system at once, due to a lack of functional style. There are few enforced boundaries between subsystems in this type of code. Like most things, moderation is key. I only introduce abstractions when there is a meaningful value-add to doing so. This is somewhat subjective, but there is a right level of application of design patterns. Not too much, nor too little.

tarsinge 7y ago
I think you can under engineer and still use a pragmatic functional style. Also yes when it’s under engineered you usually have to touch multiple parts to make a change, but in the end it usually take less time (testing for bugs included since the code is simpler). Edit: I don’t say under engineering is perfect, but compared to an over engineered code base I prefer the first for maintainability.
- JMTQp8lwXL 7y ago
  In theory, I agree with you, but in practice, I rarely see under-engineered code that is functional in style. My conjecture is that people under-engineering aren't even aware of the concepts of imperative vs. functional style. The more we can think of code as existing in a sandbox that knows nothing about the surrounding system it's in, the better. Because when you need to change that code, you don't need to juggle comprehension of the rest of the system as you edit that code. It makes reasoning about the code much simpler. This is why I agree -- code can be pragmatic and functional. But if you tend to practice having code existing in little sandboxes, you are essentially practicing design. This might be semantics, but now you have sub-systems with well-defined borders that interact with other sub-systems in proscribed, expected ways. I would consider that, to some degree, the application of design patterns and architectural concepts -- even if it's only the inevitable consequence of writing pragmatic, functional code.
- bayindirh 7y ago
  > I think you can under engineer and still use a pragmatic functional style. I actually believe this, and tried in some occasions for my personal projects. When you develop the code further, you need to touch everywhere. It starts very neat and being extremely tidy at first, but when you need to add some significant features, the design breaks down pretty badly. Then, to save the architecture, some significant refactoring runs are required, and boy, these refactorizations are expensive both time and design-wise. If your project will not add new features over time and relatively compact, under-engineering can deliver pretty nice results. Otherwise, designing with some headroom and flexibility goes a long way.
stinos 7y ago
In my own experience, it's really a must to go through both over- and under-engineering phase yourself: only then I found I got a really clear view on what's good and bad in approach, allowing me to default to somewhere in between (with outliers when useful) and in any case with focus on simplicity (but again, if needed for e.g. performance that can be violated). All of this is the basic 'learn from your mistakes' principle in the end - and it works.
- fnord123 7y ago
  I don't even think it's a phase. From what I see in my own experience, overengineering comes from not having a deadline to deliver an actual working piece of software. Unclear requirements. Fear that the code must be perfect against change without knowing what the changes might be. Underengineering comes from confidence that the issues are too small to worry about. Or the time constraints mean I can't try to find good primitives to make something robust. And if I have to return to it because it's so important, I can re-engineer it then.
  - tluyben2 7y ago
    I have been writing software professionally for almost 30 years now and yes, this is exactly what I see too. I find the underengineering far more productive though; because of time constraints I hack something together which works but is brittle (decades experience do help fight the brittleness even in a hurry though) and I do revisit to fix and refactor if the software survives.
    - fnord123 7y ago
      Underengineering is far more productive but not always possible when you're defining the overarching architecture of a multi team project. Then you have to rely on experience to do things like definelibg interfaces that can be extended later on instead of overspecifying types. It's hard to balance it just right.
  - stinos 7y ago
    overengineering comes from not having a deadline This really resonates: at the height of the phase I spent about 2 years once, half-time developing something new for which there was essentally no deadline. I went totally berserk doing insane stuff like implementing design patterns using templated 'reusable' classes in C++ so whenever something even remotely looked like a pattern I had a class for, I would use that class. Which then needed small fixes of course. And extra logging. Pluggabble. Etc, you get the point :) In the end everything was dropped because I found a much better, much simpler, existing way to do everything though. But didn't happen to me again afterwards, or at least not to that extent: it was really like having had a severe electric shock; once, but never again, before even coming close to any hot wire my brain is already subconsciously screaming to get away from the thing a.s.a.p.
EGreg 7y ago
In general I have found — over 20 years of experience architecting software - the following: 1. Platforms and reusable frameworks should be architected as well as possible. Apps can be whatever. 2. A developer who writes clean code and documents it is far better than a “10x” developer, unless you have budget for only one developer. 3. Functions should have extensibility, put the required parameters as parameters and always include an “options” at the end. Each function can have defaults that you can extend, which means you need a deep-extend method: https://qbix.com/platform/guide/javascript#functions https://qbix.com/platform/guide/javascript#functions 4. When in doubt whether to do convention A or B, take a bit more time to do C which can handle both, and add the convention in a config. You never know when someone may need something else! 5. Similar to 4, if you can have an extra indirection, add it. So you can let others hook into “before” and “after” hooks at any point. Use events instead of functions. 6. In fact try to have event driven architecture rather than futzing around with mutating data. The easiest way to sync is to maintain a linear total order for events. 7. Think about how lookups will proceed and partition everything by those keys. Sometimes you need to have duplicate tables and keep a sync from a “primary” table to a “secodary” one in the app layer. Doing this allows you to do sharding or even go serverless peer-to-peer later! 8. Security: more checks are better than less. Pile on private keys, bearer tokens (api or cookie), and so on. Use a device keychain: https://qbix.com/blog/2018/01/17/modern-security-practices-for-web-developers/ https://qbix.com/blog/2018/01/17/modern-security-practices-f...
- igor47 7y ago
  This list is full of hilariously terrible advice. Was that intentional? On the internet, nobody know if you're a dog or being sarcastic, and I dread stumbling across a code base where someone took some of this stuff seriously. Examples: > if you can have an extra indirection, add it. > Use events instead of functions. > You never know when someone may need something else!
  - EGreg 7y ago
    Please be specific about the issues and let’s discuss. I am serious - this list is optimized for maintainability of code. Developer time is more valuable than processor cycles, in most cases. Unless you are the kind of person who would argue that C++ introducing object orientation and virtual methods made everything slower and that extra indirection by default is hilariously bad architecture?
jamesb93 7y ago
You should clarify that your experience us pretty much exclusive to web development which is bot indicative of a lot of work that goes on in software creation and computer science research. Just a weird nit pick of mine that most people here seem to be web devs who assume everyone else is - it creates a weird environment for discussion.
xcubic 7y ago
From someone doing webdev but without studies, is there a small list of simple data structures or algorithms that you would recomend?

tarsinge 7y ago

I think you can under engineer and still use a pragmatic functional style. Also yes when it’s under engineered you usually have to touch multiple parts to make a change, but in the end it usually take less time (testing for bugs included since the code is simpler). Edit: I don’t say under engineering is perfect, but compared to an over engineered code base I prefer the first for maintainability.

JMTQp8lwXL 7y ago
In theory, I agree with you, but in practice, I rarely see under-engineered code that is functional in style. My conjecture is that people under-engineering aren't even aware of the concepts of imperative vs. functional style. The more we can think of code as existing in a sandbox that knows nothing about the surrounding system it's in, the better. Because when you need to change that code, you don't need to juggle comprehension of the rest of the system as you edit that code. It makes reasoning about the code much simpler. This is why I agree -- code can be pragmatic and functional. But if you tend to practice having code existing in little sandboxes, you are essentially practicing design. This might be semantics, but now you have sub-systems with well-defined borders that interact with other sub-systems in proscribed, expected ways. I would consider that, to some degree, the application of design patterns and architectural concepts -- even if it's only the inevitable consequence of writing pragmatic, functional code.
bayindirh 7y ago
> I think you can under engineer and still use a pragmatic functional style. I actually believe this, and tried in some occasions for my personal projects. When you develop the code further, you need to touch everywhere. It starts very neat and being extremely tidy at first, but when you need to add some significant features, the design breaks down pretty badly. Then, to save the architecture, some significant refactoring runs are required, and boy, these refactorizations are expensive both time and design-wise. If your project will not add new features over time and relatively compact, under-engineering can deliver pretty nice results. Otherwise, designing with some headroom and flexibility goes a long way.

stinos 7y ago

In my own experience, it's really a must to go through both over- and under-engineering phase yourself: only then I found I got a really clear view on what's good and bad in approach, allowing me to default to somewhere in between (with outliers when useful) and in any case with focus on simplicity (but again, if needed for e.g. performance that can be violated). All of this is the basic 'learn from your mistakes' principle in the end - and it works.

fnord123 7y ago
I don't even think it's a phase. From what I see in my own experience, overengineering comes from not having a deadline to deliver an actual working piece of software. Unclear requirements. Fear that the code must be perfect against change without knowing what the changes might be. Underengineering comes from confidence that the issues are too small to worry about. Or the time constraints mean I can't try to find good primitives to make something robust. And if I have to return to it because it's so important, I can re-engineer it then.
- tluyben2 7y ago
  I have been writing software professionally for almost 30 years now and yes, this is exactly what I see too. I find the underengineering far more productive though; because of time constraints I hack something together which works but is brittle (decades experience do help fight the brittleness even in a hurry though) and I do revisit to fix and refactor if the software survives.
  - fnord123 7y ago
    Underengineering is far more productive but not always possible when you're defining the overarching architecture of a multi team project. Then you have to rely on experience to do things like definelibg interfaces that can be extended later on instead of overspecifying types. It's hard to balance it just right.
- stinos 7y ago
  overengineering comes from not having a deadline This really resonates: at the height of the phase I spent about 2 years once, half-time developing something new for which there was essentally no deadline. I went totally berserk doing insane stuff like implementing design patterns using templated 'reusable' classes in C++ so whenever something even remotely looked like a pattern I had a class for, I would use that class. Which then needed small fixes of course. And extra logging. Pluggabble. Etc, you get the point :) In the end everything was dropped because I found a much better, much simpler, existing way to do everything though. But didn't happen to me again afterwards, or at least not to that extent: it was really like having had a severe electric shock; once, but never again, before even coming close to any hot wire my brain is already subconsciously screaming to get away from the thing a.s.a.p.

EGreg 7y ago

In general I have found — over 20 years of experience architecting software - the following: 1. Platforms and reusable frameworks should be architected as well as possible. Apps can be whatever. 2. A developer who writes clean code and documents it is far better than a “10x” developer, unless you have budget for only one developer. 3. Functions should have extensibility, put the required parameters as parameters and always include an “options” at the end. Each function can have defaults that you can extend, which means you need a deep-extend method: https://qbix.com/platform/guide/javascript#functions https://qbix.com/platform/guide/javascript#functions 4. When in doubt whether to do convention A or B, take a bit more time to do C which can handle both, and add the convention in a config. You never know when someone may need something else! 5. Similar to 4, if you can have an extra indirection, add it. So you can let others hook into “before” and “after” hooks at any point. Use events instead of functions. 6. In fact try to have event driven architecture rather than futzing around with mutating data. The easiest way to sync is to maintain a linear total order for events. 7. Think about how lookups will proceed and partition everything by those keys. Sometimes you need to have duplicate tables and keep a sync from a “primary” table to a “secodary” one in the app layer. Doing this allows you to do sharding or even go serverless peer-to-peer later! 8. Security: more checks are better than less. Pile on private keys, bearer tokens (api or cookie), and so on. Use a device keychain: https://qbix.com/blog/2018/01/17/modern-security-practices-for-web-developers/ https://qbix.com/blog/2018/01/17/modern-security-practices-f...

igor47 7y ago
This list is full of hilariously terrible advice. Was that intentional? On the internet, nobody know if you're a dog or being sarcastic, and I dread stumbling across a code base where someone took some of this stuff seriously. Examples: > if you can have an extra indirection, add it. > Use events instead of functions. > You never know when someone may need something else!
- EGreg 7y ago
  Please be specific about the issues and let’s discuss. I am serious - this list is optimized for maintainability of code. Developer time is more valuable than processor cycles, in most cases. Unless you are the kind of person who would argue that C++ introducing object orientation and virtual methods made everything slower and that extra indirection by default is hilariously bad architecture?

jamesb93 7y ago

You should clarify that your experience us pretty much exclusive to web development which is bot indicative of a lot of work that goes on in software creation and computer science research. Just a weird nit pick of mine that most people here seem to be web devs who assume everyone else is - it creates a weird environment for discussion.

xcubic 7y ago

From someone doing webdev but without studies, is there a small list of simple data structures or algorithms that you would recomend?