4 ms·

I fell for the everything must be a microservice / distributed across as many servers as possible trap. Even though I've read so many warnings about it here on

by adevx 5y ago

I fell for the everything must be a microservice / distributed across as many servers as possible trap. Even though I've read so many warnings about it here on HA and knew upfront I might have to rewind everything. The setup : - Distributed file system using GlusterFS - DNS load balancing using Amazon Route 53 - PostgreSQL HA clusters using Patroni - A WireGuard mesh topology between all instances. Even though it was much fun setting this all up, the ballooning complexity of all this outpaced the benefits and it felt more fragile than where I came from. I decided to just scale vertically and keep it simple. Most of my refactoring these days is coming up with more simple, more manageable solutions.

api 5y ago
> Even though it was so much fun That’s the root of quite a bit of evil in software. A closely related motive is a desire to show off. Play is good for learning but not for production systems. You’ll regret it later when you are up at 4am on a Sunday morning troubleshooting some Byzantine stack. The antidote is the realization that simplicity is harder than complexity. Simple but highly effective systems are the ones that should inspire awe and admiration. Complexity is a sign of an immature design or a lack of high level conceptual thinking. A system should be only complex enough to capture essential complexity (problem domain requirements) and no more. One more thing… there is a ton of submarine marketing in our industry that encourages complexity because it’s profitable for vendors and cloud providers. The design you outline leads to more cloud resource consumption, more lock in to cloud platforms, and eventually a need for lots of service mesh, config management, and orchestration products that if the project grows will eventually start costing money.
- dominotw 5y ago
  > That’s the root of quite a bit of evil in software. A closely related motive is a desire to show off. Yes but its also driven by keeping oneself employable. " I built a simple system to do X" is not a winning conversation in an interview. GP's complex system is what ppl say they built.
  - papito 5y ago
    I don't think that this sort of selfish thinking is dominant. The explanation is much more simple - many in our field see complexity as a virtue, not an enemy.
  - api 5y ago
    That's not how you frame it. You don't say simple, you say efficient. "I built a highly efficient system to do X that required only $$$ in monthly cloud spend and handled NN operations per second..." You can also point out how fast you built it, since simpler systems often take less time.
- namaria 5y ago
  The more I think about it the more it seems to me 'simplicity' is just another name for 'encapsulated/isolated/modularised complexity'
  - api 5y ago
    This is only true if all the complexity is necessary and intrinsic to the problem. Simplicity means there is as little incidental (unnecessary) complexity as possible.
    - namaria 5y ago
      Simplicity is a name for isolated complexity only if there is as little unnecessary complexity as possible? You can only contain intentional complexity. I think you are agreeing with me.
      - api 5y ago
        I think you're underestimating the amount of inessential unnecessary complexity in most software and systems designs. There is a lot that can be trimmed without touching required complexity.
        namaria 5y ago
        Did I say anything contrary to what you just stated at any point? I literally said simplicity is an illusion, just a name for tamed complexity.

by adevx 5y ago

api 5y ago

> Even though it was so much fun That’s the root of quite a bit of evil in software. A closely related motive is a desire to show off. Play is good for learning but not for production systems. You’ll regret it later when you are up at 4am on a Sunday morning troubleshooting some Byzantine stack. The antidote is the realization that simplicity is harder than complexity. Simple but highly effective systems are the ones that should inspire awe and admiration. Complexity is a sign of an immature design or a lack of high level conceptual thinking. A system should be only complex enough to capture essential complexity (problem domain requirements) and no more. One more thing… there is a ton of submarine marketing in our industry that encourages complexity because it’s profitable for vendors and cloud providers. The design you outline leads to more cloud resource consumption, more lock in to cloud platforms, and eventually a need for lots of service mesh, config management, and orchestration products that if the project grows will eventually start costing money.

dominotw 5y ago
> That’s the root of quite a bit of evil in software. A closely related motive is a desire to show off. Yes but its also driven by keeping oneself employable. " I built a simple system to do X" is not a winning conversation in an interview. GP's complex system is what ppl say they built.
- papito 5y ago
  I don't think that this sort of selfish thinking is dominant. The explanation is much more simple - many in our field see complexity as a virtue, not an enemy.
- api 5y ago
  That's not how you frame it. You don't say simple, you say efficient. "I built a highly efficient system to do X that required only $$$ in monthly cloud spend and handled NN operations per second..." You can also point out how fast you built it, since simpler systems often take less time.
namaria 5y ago
The more I think about it the more it seems to me 'simplicity' is just another name for 'encapsulated/isolated/modularised complexity'
- api 5y ago
  This is only true if all the complexity is necessary and intrinsic to the problem. Simplicity means there is as little incidental (unnecessary) complexity as possible.
  - namaria 5y ago
    Simplicity is a name for isolated complexity only if there is as little unnecessary complexity as possible? You can only contain intentional complexity. I think you are agreeing with me.
    - api 5y ago
      I think you're underestimating the amount of inessential unnecessary complexity in most software and systems designs. There is a lot that can be trimmed without touching required complexity.
      - namaria 5y ago
        Did I say anything contrary to what you just stated at any point? I literally said simplicity is an illusion, just a name for tamed complexity.