3 ms·

I don't know if I agree with that. Most of the complexity and bugs I see in software are not because of the problem domain, but rather because of over-abstrac

by ratww 6y ago

I don't know if I agree with that. Most of the complexity and bugs I see in software are not because of the problem domain, but rather because of over-abstraction, under-abstraction and abstraction leaks, and also because of limitations and complexities introduced by the programming model or environment. (unless you consider that "supporting five operating systems and the language must be X" is part of "essential complexity") Of course, the more complex your domain is, the bigger is the program. But the non-essential complexity that exists due to the bureaucracy of languages/libraries/frameworks is a much bigger factor in adding complexity, bug and lines of code. Some examples: - Manual allocation and deallocation of memory is a good example of something that we might think as essential, since it's intertwined with our domain code, but turns out to be unnecessary (even though the replacement has downsides). The billion-dollar problem (nulls) is another one. - Supporting multiple environments/browsers/platforms. Competition is good, but the cost is steep for third-party developers. Using multiplatform frameworks partially solves but also has drawbacks: performance, limitations, bugs, leaky abstractions. If you need to be closer to metal, then different OSs have different threading models or API styles. Sometimes they don't even expose the main loop to you. You need to work around those limitations. - In most environments we still don't have a nice way of handling async operations without leaking the abstraction. The current solution is adding "isLoading" data everywhere (or creating some abstraction around both buttons and the fetching mechanism). Concurrent Mode in React is probably the best thing we have so far. - Most modern Javascript toolchains need multiple parsing steps: in the transpiler (to convert to ES5), in the bundler (to detect dependencies), in the linter, in the prettifier, and in tests. Compatibility between them is not guaranteed, and you might get conflicts which have to be solved by finding a middle ground, and that sometimes take more time than writing features. - Dogmatism is another issue. I remember in one workplace years ago there was a "ORM only" rule and most of us would work the SQL and then convert to Rails ActiveRecord (or worse: Arel). In the end it was a complete waste of time and the results were impossible to maintain. - I also think that the old Peter Norvig quote that "design patterns are missing language features" still stands. Go has proven that it's possible to have "dumb, simple code", but in other languages our best practices involve adding non-essential complexity to products. The only exception to that in my experience is SQL: if a query is too big is not due to some bureaucracy of the language, but rather due to the complexity of the domain itself.

justapassenger 6y ago
For me what you're describing is more mess than complexity. I see your point tho, but I'd also say that some of things you describe here are direct result of real world problem domain being complex, and changing over time. For example, wrong abstractions. As long as engineers writing the software are competent (if they aren't - that's a totally different story) they'll try to chose right level of abstractions for current understanding of the problem. Years down the road, a lot of their choices maybe end up to be a mistake, very often because understanding of the problem changed, or problems itself changed, and something that was nice and clean solution isn't one anymore. If problem isn't fully fixed and 100% understood, you'll never be able to make all the right, future-proof decisions about right abstractions.
- ratww 6y ago
  Yes, it is "mess". But it's also additional complexity that is non-essential to the problem. It's what Fred Brooks called accidental complexity in his No Silver Bullet essay. It is important for us to acknowledge and differentiate between the two kinds of complexity, because we in the software industry can and should aim at reducing the accidental/non-essential type. About abstraction, even when it's good, it still depends. For example: you can use a Visitor Pattern to solve a collision detection problem, but if the language had multiple dispatch you would have less complexity. So it is definitely non-essential complexity.
  - bobthebuilders 6y ago
    But multiple dispatch could also be argued as non-essential complexity... I would much prefer a visitor pattern for the abstraction benefits, for example.
    - ratww 6y ago
      But that's not the point, that's not an example to be picked apart. The point is if there's a less complex way to solve the problem, then it's still causing non-essential (aka accidental) complexity. Even if the simpler solution is ugly, the more complex one is still introducing non-essential complexity. If we claim that every single thing we do with the code is part of the "real world problem", then we're throwing up our hands in the air and giving up on the problem of complexity of software. It's a naive and simple justification that's very comfortable for us, but doesn't amount to much. I really recommend reading the No Silver Bullet paper and Mythical Man Month. Fred Brooks explains those things very well.

by ratww 6y ago

justapassenger 6y ago

For me what you're describing is more mess than complexity. I see your point tho, but I'd also say that some of things you describe here are direct result of real world problem domain being complex, and changing over time. For example, wrong abstractions. As long as engineers writing the software are competent (if they aren't - that's a totally different story) they'll try to chose right level of abstractions for current understanding of the problem. Years down the road, a lot of their choices maybe end up to be a mistake, very often because understanding of the problem changed, or problems itself changed, and something that was nice and clean solution isn't one anymore. If problem isn't fully fixed and 100% understood, you'll never be able to make all the right, future-proof decisions about right abstractions.

ratww 6y ago
Yes, it is "mess". But it's also additional complexity that is non-essential to the problem. It's what Fred Brooks called accidental complexity in his No Silver Bullet essay. It is important for us to acknowledge and differentiate between the two kinds of complexity, because we in the software industry can and should aim at reducing the accidental/non-essential type. About abstraction, even when it's good, it still depends. For example: you can use a Visitor Pattern to solve a collision detection problem, but if the language had multiple dispatch you would have less complexity. So it is definitely non-essential complexity.
- bobthebuilders 6y ago
  But multiple dispatch could also be argued as non-essential complexity... I would much prefer a visitor pattern for the abstraction benefits, for example.
  - ratww 6y ago
    But that's not the point, that's not an example to be picked apart. The point is if there's a less complex way to solve the problem, then it's still causing non-essential (aka accidental) complexity. Even if the simpler solution is ugly, the more complex one is still introducing non-essential complexity. If we claim that every single thing we do with the code is part of the "real world problem", then we're throwing up our hands in the air and giving up on the problem of complexity of software. It's a naive and simple justification that's very comfortable for us, but doesn't amount to much. I really recommend reading the No Silver Bullet paper and Mythical Man Month. Fred Brooks explains those things very well.