7 ms·

Complexity is like energy, it cannot be created or destroyed, it is a fundamental property of any process or thing. However, complexity can be managed. Humans

Complexity is like energy, it cannot be created or destroyed, it is a fundamental property of any process or thing. However, complexity can be managed. Humans do this using abstraction as a tool. We divide the complex problem up into a sequence of several simpler states, and we find it easier to understand the simpler problems along with the sequence within which they lie. Good software uses this same approach to reduce complex issues to a manageable process. A good tool makes the simple things easy and the complex thing possible, the design of the tool reflects the effort and work that the designer out in to u the problem they’re trying to solve, and to produce something that helps guide others along that self-same path of understanding, without them having to put in the same level of effort; it establishes the golden path through the marshes and bogs of difficulties that the problem domain throws up. “Embracing complexity” is a measure of last resort, IMHO. It means the tool developer could not analyze the problem and come up with a good solution; it means “here, you figure it out”; it means giving up on one of the fundamental reasons for the tools existence. Sometimes, embracing complexity and the ensuing struggle that this necessitates is simply what you have to do, but not often. Maybe, maybe this is one of those times, but I always start off with a critical eye when someone tells me that a complicated thing is “the only way it can be done”. Colour me sceptical.

crimsonalucard 7y ago
In a certain sense complexity CAN be destroyed. Yes there is a minimal level of complexity that must exist, but given an entity with a purpose, in many cases it is possible to reduce the complexity of that entity without changing its' purpose. Take for example this function which adds one to an integer: function addOne(x: int) -> int { return (x * 1) + 0 - x + x + 1; } can be reduced to: function addOne(x: int) -> int { return x + 1; } The examples are obvious but from the examples you can deduce that their are many instances in software engineering where such reductions can exist but are not obvious at all.
- jerf 7y ago
  I phrase it as "The solution can be as simple as the problem but no simpler", which means, you can "create" complexity. We see this all the time in code, after all, we must account for this. But you can't make the solution simpler than the problem. There's kind of a "by definition" aspect to this, so it's kinda circular, but it's still a valuable observation, because we also see this in code all the time. For instance, it plays a significant role in the leakiness of abstractions; that leakiness is typically the attempt to make a solution that is simpler than the entire problem space. You can be fine as long as your particular problem fits in the space of the leaky solutions, but once your problem pokes out of that you can be in big trouble.
  - davidw 7y ago
    Yes! > you can "create" complexity AKA "the complicator's gloves" https://thedailywtf.com/articles/The_Complicator_0x27_s_Gloves https://thedailywtf.com/articles/The_Complicator_0x27_s_Glov...
  - crimsonalucard 7y ago
    It's not that simple. For one we don't have a formal definition of "simple" and secondly, even when we do have a definition of "simple" for certain contexts, we can never prove that something is at its' "simplest" state or not at its' "simplest" state. For example, newtons laws of motion, seemingly is a simple solution to finding a formal method of describing motion. It seems to fit the problem but there is no way to verify that it is the simplest solution to the problem and we don't even really know what "simple" is. We know through experimentation that relativity fits observations more accurately but even with relativity we don't know if it's the most "simple" solution or even if its' fully correct. Because you cannot know or ever know whether a solution is at its' simplest form, there is ALWAYS an opportunity for a reduction to be discovered. You absolutely cannot say that "The solution can be as simple as the problem but no simpler" because we don't know what "simple" is and we don't know whether or not a "simpler" solution may be discovered in the future.
    - llimllib 7y ago
      Just because it lacks a formal definition does not mean that somebody cannot say it. It's a human notion, not a formal one. (Also: I'd suggest that it's pretty rude to tell somebody they "absolutely cannot say" something, and try to avoid telling you that you absolutely cannot say that somebody absolutely cannot say something)
- spacedcowboy 7y ago
  I take your point, but the complexity I’m talking about is what remains after you’ve done any readily applicable simplifications. Any normal task has an inherent level of complexity (“add one” is a very simple task, so can be reduced down as you point out). “Invert a matrix” is somewhat less reducible. “Calculate a quaternion” even less so, etc. We generally string dozens or hundreds of these simpler tasks into sequences to get what we want, and human understanding being what it is, there’s quite a difference in conceptual complexity between “fly the plane to [here]” and doing all that maths. There’s probably a relationship to subitising here as well, the brain does seem to be able to process small groups of things, even if it knows those things represent larger concepts, better than larger groups of things. This is just speculation on my part though.
  - crimsonalucard 7y ago
    I got into a really deep conversation about this here: https://news.ycombinator.com/item?id=23042796 https://news.ycombinator.com/item?id=23042796 You may have seen it, it's in the same thread of conversation we're having. Basically there's actually axiomatic proof that you can never know if you applied all possible applicable simplifications. Therefore you can never know whether or not more complexity can be destroyed or reduced.
- jes 7y ago
  With respect, I’m not seeing how your first addOne example returns an integer one greater than its integer argument. Am I missing something?
  - crimsonalucard 7y ago
    (x * 1) + 0 - x + x + 1 x + 0 - x + x + 1 x - x + x + 1 0 + x + 1 x + 1
timw4mail 7y ago
I see embracing complexity as acknowledging it exists, and attempting to represent the complexity in parts and layers. There are always cross-cutting concerns that aren't easily transformed, or become that way due to code changes. I think this area is where managing complexity is most difficult.
ninjapenguin54 7y ago
Complexity can easily be manufactured. Comparing it to something as fundamental as energy is pure bollocks. Heres an amusing and simple example of manufactured complexity: https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpriseEdition https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
- bproctor 7y ago
  I think the analogy to energy is pretty good. Like the FizzBuzz thing, you can also build a machine that has a lot of energy but doesn't accomplish anything useful.
  - mannykannot 7y ago
    The conservation of energy, specifically, is a misleading analogy. If you want an energy-related analogy, I think you will find a better one in entropy (especially given its close relationship to information.)
- majkinetor 7y ago
  That is not exactly how you claim it to be. Introduce enterprise into any system and complexity goes over the roof ASAP - suddenly its not only your machine but TEAM of people, history, dashboards with metrics, logs, automation, security etc... This one isn't even approaching any of it except dev side and CI.
- pron 7y ago
  If only there were some scientific discipline that studied complexity and proved that problems possess some essential, minimal, complexity below which no implementation can go... If anyone proved such a result, I'm sure they would be awarded a prize of some kind.
  - MaxBarraclough 7y ago
    But that's a whole different use of the word 'complexity'. Bubblesort has inferior time complexity than Timsort, but is simpler. The bounds of the comparative sort problem doesn't tell us that either.
ashtonkem 7y ago
Manufacturing complexity is absolutely doable, have you never heard of a Rube Goldberg machine?
taneq 7y ago
Complexity is like entropy, not energy. It increases, unless you spend a significant amount of energy removing it from a small area, and in the process increase the complexity somewhere else by a larger amount. However, any degree of functionality requires at least a matching degree of complexity. In that sense, some complexity is 'essential' and that complexity does need to live somewhere in your project.
draw_down 7y ago
It absolutely can be created, believe it.

crimsonalucard 7y ago

In a certain sense complexity CAN be destroyed. Yes there is a minimal level of complexity that must exist, but given an entity with a purpose, in many cases it is possible to reduce the complexity of that entity without changing its' purpose. Take for example this function which adds one to an integer: function addOne(x: int) -> int { return (x * 1) + 0 - x + x + 1; } can be reduced to: function addOne(x: int) -> int { return x + 1; } The examples are obvious but from the examples you can deduce that their are many instances in software engineering where such reductions can exist but are not obvious at all.

jerf 7y ago
I phrase it as "The solution can be as simple as the problem but no simpler", which means, you can "create" complexity. We see this all the time in code, after all, we must account for this. But you can't make the solution simpler than the problem. There's kind of a "by definition" aspect to this, so it's kinda circular, but it's still a valuable observation, because we also see this in code all the time. For instance, it plays a significant role in the leakiness of abstractions; that leakiness is typically the attempt to make a solution that is simpler than the entire problem space. You can be fine as long as your particular problem fits in the space of the leaky solutions, but once your problem pokes out of that you can be in big trouble.
- davidw 7y ago
  Yes! > you can "create" complexity AKA "the complicator's gloves" https://thedailywtf.com/articles/The_Complicator_0x27_s_Gloves https://thedailywtf.com/articles/The_Complicator_0x27_s_Glov...
- crimsonalucard 7y ago
  It's not that simple. For one we don't have a formal definition of "simple" and secondly, even when we do have a definition of "simple" for certain contexts, we can never prove that something is at its' "simplest" state or not at its' "simplest" state. For example, newtons laws of motion, seemingly is a simple solution to finding a formal method of describing motion. It seems to fit the problem but there is no way to verify that it is the simplest solution to the problem and we don't even really know what "simple" is. We know through experimentation that relativity fits observations more accurately but even with relativity we don't know if it's the most "simple" solution or even if its' fully correct. Because you cannot know or ever know whether a solution is at its' simplest form, there is ALWAYS an opportunity for a reduction to be discovered. You absolutely cannot say that "The solution can be as simple as the problem but no simpler" because we don't know what "simple" is and we don't know whether or not a "simpler" solution may be discovered in the future.
  - llimllib 7y ago
    Just because it lacks a formal definition does not mean that somebody cannot say it. It's a human notion, not a formal one. (Also: I'd suggest that it's pretty rude to tell somebody they "absolutely cannot say" something, and try to avoid telling you that you absolutely cannot say that somebody absolutely cannot say something)
spacedcowboy 7y ago
I take your point, but the complexity I’m talking about is what remains after you’ve done any readily applicable simplifications. Any normal task has an inherent level of complexity (“add one” is a very simple task, so can be reduced down as you point out). “Invert a matrix” is somewhat less reducible. “Calculate a quaternion” even less so, etc. We generally string dozens or hundreds of these simpler tasks into sequences to get what we want, and human understanding being what it is, there’s quite a difference in conceptual complexity between “fly the plane to [here]” and doing all that maths. There’s probably a relationship to subitising here as well, the brain does seem to be able to process small groups of things, even if it knows those things represent larger concepts, better than larger groups of things. This is just speculation on my part though.
- crimsonalucard 7y ago
  I got into a really deep conversation about this here: https://news.ycombinator.com/item?id=23042796 https://news.ycombinator.com/item?id=23042796 You may have seen it, it's in the same thread of conversation we're having. Basically there's actually axiomatic proof that you can never know if you applied all possible applicable simplifications. Therefore you can never know whether or not more complexity can be destroyed or reduced.
jes 7y ago
With respect, I’m not seeing how your first addOne example returns an integer one greater than its integer argument. Am I missing something?
- crimsonalucard 7y ago
  (x * 1) + 0 - x + x + 1 x + 0 - x + x + 1 x - x + x + 1 0 + x + 1 x + 1

timw4mail 7y ago

I see embracing complexity as acknowledging it exists, and attempting to represent the complexity in parts and layers. There are always cross-cutting concerns that aren't easily transformed, or become that way due to code changes. I think this area is where managing complexity is most difficult.

ninjapenguin54 7y ago

Complexity can easily be manufactured. Comparing it to something as fundamental as energy is pure bollocks. Heres an amusing and simple example of manufactured complexity: https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpriseEdition https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...

bproctor 7y ago
I think the analogy to energy is pretty good. Like the FizzBuzz thing, you can also build a machine that has a lot of energy but doesn't accomplish anything useful.
- mannykannot 7y ago
  The conservation of energy, specifically, is a misleading analogy. If you want an energy-related analogy, I think you will find a better one in entropy (especially given its close relationship to information.)
majkinetor 7y ago
That is not exactly how you claim it to be. Introduce enterprise into any system and complexity goes over the roof ASAP - suddenly its not only your machine but TEAM of people, history, dashboards with metrics, logs, automation, security etc... This one isn't even approaching any of it except dev side and CI.
pron 7y ago
If only there were some scientific discipline that studied complexity and proved that problems possess some essential, minimal, complexity below which no implementation can go... If anyone proved such a result, I'm sure they would be awarded a prize of some kind.
- MaxBarraclough 7y ago
  But that's a whole different use of the word 'complexity'. Bubblesort has inferior time complexity than Timsort, but is simpler. The bounds of the comparative sort problem doesn't tell us that either.

ashtonkem 7y ago

Manufacturing complexity is absolutely doable, have you never heard of a Rube Goldberg machine?

taneq 7y ago

Complexity is like entropy, not energy. It increases, unless you spend a significant amount of energy removing it from a small area, and in the process increase the complexity somewhere else by a larger amount. However, any degree of functionality requires at least a matching degree of complexity. In that sense, some complexity is 'essential' and that complexity does need to live somewhere in your project.

draw_down 7y ago

It absolutely can be created, believe it.